Implementing image dehazing and defogging on a Field Programmable Gate Array (FPGA) offers efficiency. Dehazing an image becomes particularly challenging in the presence of fog or haze. However, employing a dark chann...
详细信息
A vital and rapidly growing application, remote sensing offers vast yet sparsely labeled, spatially aligned multimodal data;this makes self-supervised learning algorithms invaluable. We present CROMA: a framework that...
详细信息
ISBN:
(纸本)9781713899921
A vital and rapidly growing application, remote sensing offers vast yet sparsely labeled, spatially aligned multimodal data;this makes self-supervised learning algorithms invaluable. We present CROMA: a framework that combines contrastive and reconstruction self-supervised objectives to learn rich unimodal and multimodal representations. Our method separately encodes masked-out multispectral optical and synthetic aperture radar samples-aligned in space and time-and performs cross-modal contrastive learning. Another encoder fuses these sensors, producing joint multimodal encodings that are used to predict the masked patches via a lightweight decoder. We show that these objectives are complementary when leveraged on spatially aligned multimodal data. We also introduce X- and 2D-ALiBi, which spatially biases our cross- and self-attention matrices. These strategies improve representations and allow our models to effectively extrapolate to images up to 17.6x larger at test-time. CROMA outperforms the current SoTA multispectral model, evaluated on: four classification benchmarks-finetuning (*** arrow 1.8%), linear (*** arrow 2.4%) and nonlinear (*** arrow 1.4%) probing, kNN classification (*** arrow 3.5%), and K-means clustering (*** arrow 8.4%);and three segmentation benchmarks (*** arrow 6.4%). CROMA's rich, optionally multimodal representations can be widely leveraged across remote sensing applications.
In elastic optical networks, multi-granularity channels with different bandwidth impose higher demands on the digital signal processing (DSP) algorithms of *** regular DSP approaches, the clock recovery algorithm is s...
详细信息
In aeronautics, engineering, medicine, robotics and other industries, optical methods are widely used to measure the geometry and surface deformation of various objects from their images. These methods are based on di...
详细信息
Recently, vehicle to Everything (v2X) technique has drawn great attention in various industries. Unlike traditional wireless communication systems, malicious attacks against v2X network may not only cause user privacy...
详细信息
In this article, we will analyze methods for modeling user ratings of a video sequence and their potential application to extended reality systems. The main approaches and algorithms used to evaluate the quality of a ...
详细信息
Recent technological advances have stimulated the exponential growth of social network data, driving an increase in research into sentiment analysis. Thus, studies exploring the intersection of Natural Language Proces...
详细信息
The Kerr effect and the Faraday effect are considered for studying the effects of the transverse electric field and the longitudinal magnetic field of lightning in an optical fiber. Presented are experimental study re...
详细信息
In recent years, the rapid advancement of 5G technology has brought to the forefront the pivotal role of Multiple-Input Multiple-Output (MIMO) system algorithms. This paper delves into a comprehensive exploration of t...
详细信息
Considering the challenges associated with robots in optoelectronic imaging applications, typically require real-time and accurate recognition and localization of targets, especially in complex environments. Due to th...
详细信息
暂无评论