检索结果-内蒙古大学图书馆

12th International Conference on Learning Representations, ICLR 2024

作者： Tian, Fengrui Duan, Yueqi Wang, Angtian Guo, Jianfei Du, Shaoyi National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications Institute of Artificial Intelligence and Robotics Xi'an Jiaotong University China Tsinghua University China Johns Hopkins University United States Shanghai AI Laboratory China

In this work, we pioneer Semantic Flow, a neural semantic representation of dynamic scenes from monocular videos. In contrast to previous NeRF methods that reconstruct dynamic scenes from the colors and volume densities of individual points, Semantic Flow learns semantics from continuous flows that contain rich 3D motion information. As there is 2D-to-3D ambiguity problem in the viewing direction when extracting 3D flow features from 2D video frames, we consider the volume densities as opacity priors that describe the contributions of flow features to the semantics on the frames. More specifically, we first learn a flow network to predict flows in the dynamic scene, and propose a flow feature aggregation module to extract flow features from video frames. Then, we propose a flow attention module to extract motion information from flow features, which is followed by a semantic network to output semantic logits of flows. We integrate the logits with volume densities in the viewing direction to supervise the flow features with semantic labels on video frames. Experimental results show that our model is able to learn from multiple dynamic scenes and supports a series of new tasks such as instance-level scene editing, semantic completions, dynamic scene tracking and semantic adaption on novel scenes. © 2024 12th International Conference on Learning Representations, ICLR 2024. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Quality-Improved and Property-Preserved Polarimetric Imaging via Complementarily Fusing 38

Quality-Improved and Property-Preserved Polarimetric Imaging...

引用

38th Conference on Neural information processing Systems, NeurIPS 2024

作者： Zhou, Chu Liu, Yixing Xu, Chao Shi, Boxin National Institute of Informatics Japan State Key Laboratory for Multimedia Information Processing School of CS Peking University China National Engineering Research Center of Visual Technology School of CS Peking University China National Key Laboratory of General Artificial Intelligence School of IST Peking University China

Polarimetric imaging is a challenging problem in the field of polarization-based vision, since setting a short exposure time reduces the signal-to-noise ratio, making the degree of polarization (DoP) and the angle of polarization (AoP) severely degenerated, while if setting a relatively long exposure time, the DoP and AoP would tend to be over-smoothed due to the frequently-occurring motion blur. This work proposes a polarimetric imaging framework that can produce clean and clear polarized snapshots by complementarily fusing a degraded pair of noisy and blurry ones. By adopting a neural network-based three-phase fusing scheme with specially-designed modules tailored to each phase, our framework can not only improve the image quality but also preserve the polarization properties. Experimental results show that our framework achieves state-of-the-art performance. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Hidden States in LLMs Improve EEG Representation Learning and visual Decoding 27

Hidden States in LLMs Improve EEG Representation Learning an...

引用

27th European Conference on Artificial Intelligence, ECAI 2024

作者： Liu, Aoyang Jing, Haodong Liu, Yulong Ma, Yongqiang Zheng, Nanning National Key Laboratory of Human-Machine Hybrid Augmented Intelligence China National Engineering Research Center of Visual Information and Applications China Institute of Artificial Intelligence and Robotics Xi'an Jiaotong University China The Hong Kong University of Science and Technology Hong Kong

ISBN: (纸本)9781643685489

Analyzing brain signals and reconstructing visual stimuli from the brain can facilitate further exploration on cognitive functions of the human brain, which have attracted strong interest in neuroscience and artificial intelligence. However, due to defects such as complex noises and the lack of alignment accuracy, efficient methods for extracting information from Electroencephalogram (EEG) signals are still very limited, making it difficult to perform EEG visual decoding tasks. Our study shows a way to handle the issues by proposing a new method for EEG representation learning and visual decoding, thus completing end-to-end image reconstruction tasks from EEG signals. We utilize the ability of semantic extraction and prediction of large language models (LLMs) to enhance the performance of EEG feature extraction. For semantic representation learning, we align EEG signals with target semantic embeddings, which are obtained from hidden states of Large Language Model Meta AI 2 (LLaMa-2) by inputting descriptions of images into the model. We also extract visual features from EEG signals to improve the quantity of the reconstructed images at low levels. Then we fuse semantic features and visual features by applying a pre-trained diffusion model and finally generate the corresponding images. We are the first to incorporate the LLM into EEG visual decoding tasks. Our method achieves the state-of-the-art result of EEG classification accuracy and the quality of reconstructed images on ImageNet-EEG datasets. In one word, our work is an important step forward in the field of exploiting the relationship between language models and human visual cognition. Our codes are available at https://***/lay-atsa/llm4eeg. © 2024 The Authors.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

EventAlign: LiDAR-Event Camera Calibration with Event Alignment Loss 27

EventAlign: LiDAR-Event Camera Calibration with Event Alignm...

引用

27th IEEE International Conference on Intelligent Transportation Systems, ITSC 2024

作者： Dong, Pengfei Hai, Renwei Shen, Yanqing Zhu, Ziyu Chen, Shitao Xin, Jingmin Zheng, Nanning Institute of Artificial Intelligence and Robotics Xi'an Jiaotong University National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications Shaanxi Xi'an710049 China Shunan Academy of Artificial Intelligence Zhejiang Ningbo China

ISBN: (纸本)9798331505929

In recent years, event cameras, as a type of bio-inspired sensor, have become increasingly important in the fields of robotics and autonomous driving. Given that events inherently lack depth information, a multi-sensor solution is applied to combine events and LiDAR scans, and its performance highly depends on precise calibration between different sensors. However, due to the distinctive representation of event cameras, which only capture dynamic objects without texture structure, most previous calibration methods for event cameras have complex setups and controlled lighting conditions. In this paper, we propose a novel extrinsic calibration method by simulating events with LiDAR, which does not require any equipment beyond the sensors themselves. Specifically, we first extract LiDAR events from the continuous raw point clouds. Then a clustering algorithm is applied to refine this event data and identify motion centroids. Finally, a two-step optimization process is proposed: an initial rough calibration based on these centroids and a subsequent fine calibration achieved through nearest neighbor matching between LiDAR events and camera events. The experimental results validate that our algorithm achieves high accuracy and robustness in simulations and real-world experiments. © 2024 IEEE.

关键词： Nearest neighbor search

来源：评论

学校读者我要写书评

暂无评论

Improved Capon Estimator for High-Resolution DOA Estimation and Its Statistical Analysis

引用

IEEE/CAA Journal of Automatica Sinica 2023年第8期10卷 1716-1729页

作者： Weiliang Zuo Jingmin Xin Changnong Liu Nanning Zheng Akira Sano IEEE the National Key Laboratory of Human-Machine Hybrid Augmented Intelligence the National Engineering Research Center for Visual Information and Applicationsand the Institute of Artificial Intelligence and RoboticsXi’an Jiaotong UniversityXi’an 710049China the Data Center China Construction BankBeijing 100033China the Department of System Design Engineering Keio UniversityYokohama 223-8522Japan

Despite some efforts and attempts have been made to improve the direction-of-arrival(DOA)estimation performance of the standard Capon beamformer(SCB)in array processing,rigorous statistical performance analyses of these modified Capon estimators are still *** paper studies an improved Capon estimator(ICE)for estimating the DOAs of multiple uncorrelated narrowband signals,where the higherorder inverse(sample)array covariance matrix is used in the Capon-like cost *** establishing the relationship between this nonparametric estimator and the parametric and classic subspace-based MUSIC(multiple signal classification),it is clarified that as long as the power order of the inverse covariance matrix is increased to reduce the influence of signal subspace components in the ICE,the estimation performance of the ICE becomes equivalent to that of the MUSIC regardless of the signal-to-noise ratio(SNR).Furthermore the statistical performance of the ICE is analyzed,and the large-sample mean-squared-error(MSE)expression of the estimated DOA is *** the effectiveness and the theoretical analysis of the ICE are substantiated through numerical examples,where the Cramer-Rao lower bound(CRB)is used to evaluate the validity of the derived asymptotic MSE expression.

关键词： Capon beamformer direction-of-arrival(DOA)estimation large-sample mean-squared-error(MSE) subspace-based methods uniform linear array

来源：评论

学校读者我要写书评

暂无评论

Pest Detection and Identification Guided by Feature Maps 12

Pest Detection and Identification Guided by Feature Maps

引用

12th International Conference on Image processing Theory, Tools and applications, IPTA 2023

作者： Chen, Miao Chen, Yanan Guo, Minghui Wang, Jianji Xi'an Jiaotong University National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications Institute of Artificial Intelligence and Robotics China Xi'an Jiaotong University School of Software Engineering Xi'an China

ISBN: (纸本)9798350325416

Effective pest detection and identification are of great significance for agricultural activities, and morden machine learning methods, especially the deep neural network, undoubtedly provide convenient and effective ways for this issue. Most of the mainstream methods for pest detection require annotation of manual bounding box. However, the public pest datasets usually lack the labeling information. In this work, we propose a method for detecting pests with weak supervision. We obtain the key regions of the pest targets by deeply mining the information in the feature maps of the convolutional neural networks (CNNs). First, using the class activation mapping technique to generate activation maps of the target class during forward propagation of the model, and then obtaining the key regions of the target based on the hotspots extracted from the activation maps. In addition, our proposed method can be deployed as a general strategy to other CNNs, and experimental results demonstrate that our method can steadily improve the classification accuracy of the model. By applying our method to the single Resnet50, we can achieve 74.27% classification accuracy on the publicly accessible IP102 dataset. Another attribution of this paper is that we propose a lightweight approach to find solutions for pest detection in agricultural automation applications. © 2023 IEEE.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Color4E: Event Demosaicing for Full-color Event Guided Image Deblurring 24

Color4E: Event Demosaicing for Full-color Event Guided Image...

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Ma, Yi Duan, Peiqi Hong, Yuchen Zhou, Chu Zhang, Yu Ren, Jimmy Shi, Boxin National Key Laboratory for Multimedia Information Processing and National Engineering Research Center of Visual Technology School of Computer Science Peking University Beijing China National Key Lab of General AI School of Intelligence Science and Technology Peking University Beijing China SenseTime Research Beijing China

ISBN: (纸本)9798400706868

Neuromorphic event sensors are novel visual cameras that feature high-speed illumination-variation sensing and have found widespread application in guiding frame-based imaging enhancement. This paper focuses on color restoration in the event-guided image deblurring task, we fuse blurry images with mosaic color events instead of mono events to avoid artifacts such as color bleeding. The challenges associated with this approach include demosaicing color events for reconstructing full-resolution sampled signals and fusing bimodal signals to achieve image deblurring. To meet these challenges, we propose a novel network called Color4E to enhance the color restoration quality for the image deblurring task. Color4E leverages an event demosaicing module to upsample the spatial resolution of mosaic color events and a cross-encoding image deblurring module for fusing bimodal signals, a refinement module is designed to fuse full-color events and refine initial deblurred images. Furthermore, to avoid the real-simulated gap of events, we implement a display-filter-camera system that enables mosaic and full-color event data captured synchronously, to collect a real-captured dataset used for network training and validation. The results on the public dataset and our collected dataset show that Color4E enables high-quality event-based image deblurring compared to state-of-the-art methods. © 2024 ACM.

关键词： Restoration

来源：评论

学校读者我要写书评

暂无评论

Return of Small-Scale Crowd Counting via Fast and Accurate Semi-Supervised Least Squares Model

Return of Small-Scale Crowd Counting via Fast and Accurate S...

引用

2023 IEEE Symposium Series on Computational Intelligence, SSCI 2023

作者： Luo, Hao Du, Shaoyi Tian, Zhiqiang School of Software Engineering Xi'an Jiaotong University Xi'an China Institute of Artificial Intelligence and Robotics National Engineering Research Center for Visual Information and Applications Xi'an Jiaotong University National Key Laboratory of Human-Machine Hybrid Augmented Intelligence Xi'an China

ISBN: (纸本)9781665430654

Existing crowd counting techniques have achieved significant progress with the emergence of deep learning. During development, emerging crowd counting methods have generally become more and more complex and enormous, enabling them to understand and process more prior knowledge from input data. However, they suffer from two major drawbacks: 1) they generally require a significant amount of labeled training samples, which is labor-intensive, and 2) they require increasing computational hardware resources, making it luxurious and impractical to apply directly in small-scale scenes. To address these issues, we formulate crowd counting as a classification problem and leverage least squares model with a novel semi-supervised strategy. Technically, we construct the least squares model based on only two regularization terms: a regression term and a discriminative relaxation term. Moreover, we propose a semi-supervised soft label correcting strategy incorporated in the model. As a result, a fast and accurate crowd counting method is achieved. Experimental results on five small-scale benchmarks demonstrate the proposed method outperforms the other competitors in terms of both regression metrics and consumed time. © 2023 IEEE.

关键词： classification crowd counting least squares model semi-supervised

来源：评论

学校读者我要写书评

暂无评论

VIPNet: Combining Viewpoint information and Shape Priors for Instant Multi-view 3D Reconstruction 17th

VIPNet: Combining Viewpoint Information and Shape Priors fo...

引用

17th Asian Conference on Computer Vision, ACCV 2024

作者： Ye, Weining Li, Zhixuan Jiang, Tingting School of Computer Science Peking University Beijing100871 China College of Computing and Data Science Nanyang Technological University Singapore Singapore National Engineering Research Center of Visual Technology National Key Laboratory for Multimedia Information Processing School of Computer Science National Biomedical Imaging Center Peking University Beijing100871 China

ISBN: (纸本)9789819609680

While the multi-view 3D reconstruction task has made significant progress, existing methods simply fuse multi-view image features without effectively leveraging available auxiliary information, especially the viewpoint information for guiding and associating features of different views. To this end, we propose to enhance multi-view 3D reconstruction with the power of viewpoint information. Specifically, a simple-yet-effective viewpoint estimator is designed to learn and provide comprehensive viewpoint knowledge for locating and associating learned features from different views. Moreover, to improve the 3D reconstruction quality when 2D images of only very few viewpoints are available, we propose to learn the shape prior knowledge to provide sufficient shape information for compensating the limited 2D observations. Overall, we present VIPNet, benefiting from Viewpoint information and Shape Prior learning for high-quality multi-view 3D reconstruction. Extensive experiments validate the effectiveness of the proposed VIPNet, which achieves state-of-the-art performance on challenging datasets and shows well generalization ability in real-world scenarios. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： 3D modeling

来源：评论

学校读者我要写书评

暂无评论

A Toeplitz Prior-Based Deep Learning Framework for DOA Estimation With Unknown Mutual Coupling 31

A Toeplitz Prior-Based Deep Learning Framework for DOA Estim...

引用

31st European Signal processing Conference, EUSIPCO 2023

作者： Jiang, Zhuoqian Xin, Jingmin Zuo, Weiliang Zheng, Nanning Sano, Akira Institute of Artificial Intelligence and Robotics Xi'an Jiaotong University Xi'an710049 China National Engineering Laboratory for Visual Information Processing and Applications Xi'an Jiaotong University Xi'an710049 China Shunan Academy of Artificial Intelligence Zhejiang Ningbo315000 China Department of System Design Engineering Keio University Yokohama223-8522 Japan

ISBN: (纸本)9789464593600

In this paper, we explore the problem of direction-of-arrival (DOA) estimation with unknown mutual coupling using a deep learning (DL) framework which is based on Toeplitz prior. First, for source number estimation, we model it as a multilabel classification task and build a source number detection network (SNDN) to learn relevant information in the real sample covariance matrix. Next, taking full advantage of the Toeplitz structure, an ideal covariance reconstruction network (ICRN) is proposed to recover the ideal covariance matrix free from mutual coupling and noise interference. Furthermore, we design a database to store the parameters of ICRN after training on different numbers of sources, and its role is to load the corresponding parameters for ICRN according to the detection results of SNDN. Finally, the DOAs can be easily estimated from the restored covariance matrix by the MUSIC. The simulation results show our proposed approach not only outperforms the existing classical methods, but in some cases its DOA estimation accuracy can even exceed the Cramér-Rao Lower Bound. © 2023 European Signal processing Conference, EUSIPCO. All rights reserved.

关键词： Covariance matrix

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：