检索结果-内蒙古大学图书馆

HTML: Hybrid Temporal-scale Multimodal Learning Framework for Referring Video Object Segmentation

学校读者我要写书评

暂无评论

HTML: Hybrid Temporal-scale Multimodal Learning Framework fo...

International Conference on computer Vision (ICCV)

作者： Mingfei Han Yali Wang Zhihui Li Lina Yao Xiaojun Chang Yu Qiao ReLER AAII UTS Data61 CSIRO Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shanghai AI Laboratory Shanghai China Shandong Artificial Intelligence Qilu University of Technology Department of Computer Vision Mohamed bin Zayed University of Artificial Intelligence

Referring Video Object Segmentation (RVOS) is to segment the object instance from a given video, according to the textual description of this object. However, in the open world, the object descriptions are often diversified in contents and flexible in lengths. This leads to the key difficulty in RVOS, i.e., various descriptions of different objects are corresponding to different temporal scales in the video, which is ignored by most existing approaches with single stride of frame sampling. To tackle this problem, we propose a concise Hybrid Temporal-scale Multimodal Learning (HTML) framework, which can effectively align lingual and visual features to discover core object semantics in the video, by learning multimodal interaction hierarchically from different temporal scales. More specifically, we introduce a novel inter-scale multimodal perception module, where the language queries dynamically interact with visual features across temporal scales. It can effectively reduce complex object confusion by passing video context among different scales. Finally, we conduct extensive experiments on the widely used benchmarks, including Ref-Youtube-VOS, Ref-DAVIS17, A2D-Sentences and JHMDB-Sentences, where our HTML achieves state-of-the-art performance on all these datasets.

关键词：

FluidPlaying: Efficient Adaptive Simulation for Highly Dynamic Fluid

学校读者我要写书评

暂无评论

FluidPlaying: Efficient Adaptive Simulation for Highly Dynam...

virtual reality and 3D User Interfaces Abstracts and Workshops (VRW), IEEE Conference on

作者： Sinuo Liu Xiaojuan Ban Sheng Li Haokai Zeng Xiaokun Wang Yanrui Xu Fei Zhu Guoping Wang Graphics & Interaction Lab National Biomedical Imaging Center School of Computer Science Peking University School of Intelligence Science and Technology University of Science and Technology Beijing Beijing Engineering Technology Research Center of Virtual Simulation and Visualization Peking University Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)

We present FliudPlaying, a novel dynamic level-based spatially adaptive simulation method that can handle highly dynamic fluid efficiently. To capture the subtle detail of the fluid surface, the high-resolution simulation is performed not only at the free surface but also at those regions with high vorticity levels and velocity difference levels. To minimize the density error, an online optimization scheme is used when increasing the resolution by particle splitting. We also proposed a neighbor-based splash enhancement to compensate for the loss of dynamic details. Compared with the high-resolution simulation baseline, our method can achieve over 3× speedups while consuming only less than 10% computational resources. Furthermore, our method can make up for the loss of high-frequency details caused by the spatial adaptation, and provide more realistic dynamics in particle-based fluid simulation.

关键词： Solid modeling Adaptation models Three-dimensional displays Fluid dynamics Computational modeling Conferences virtual reality

A New Dynamic Routing Network for Monocular Depth Estimation 12

学校读者我要写书评

暂无评论

A New Dynamic Routing Network for Monocular Depth Estimation

12th International Conference on CYBER Technology in Automation, Control, and Intelligent Systems, CYBER 2022

作者： Luo, Zhehao Luo, Sijin Liang, Guoyuan Wu, Xinyu Guangdong Provincial Key Lab of Robotics and Intelligent System Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Guangdong Province Shenzhen China University of Chinese Academy of Sciences Beijing China Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems Chinese Academy of Sciences Guangdong Province Shenzhen China Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Guangdong Province Shenzhen China

ISBN: (数字)9781665472678

ISBN: (纸本)9781665472678

Depth estimation is an essential task for understanding the geometry of 3D scenes. Compared with multi-view-based methods, monocular depth estimation is more challenging for the requirement of integrating not only global information but also local cues from various parts. Recently, numerous Convolutional Neural Networks (CNNs) based approaches relying on pre-defined static architectures have been reported. Although many of them have achieved remarkable improvements in accuracy, there remains problem with the generality of the static model when it comes to diverse scenes. In this paper, a novel deep network with dynamic routing architecture is proposed with a dynamically adjustable structure with respect to different inputs, therefore leading to not only improvements on both accuracy and generality but also decrease of the model parameters. Extensive experiments of monocular depth estimation have been conducted on two well-known challenging datasets (i.e., NYUD-v2 and Make3D) as well as the mixed dataset, and have verified the effectiveness of the proposed dynamic framework. © 2022 IEEE.

关键词： Deep neural networks

Tackling Noisy labels with Network Parameter Additive Decomposition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Jingyi Xia, Xiaobo Lan, Long Wu, Xinghao Yu, Jun Yang, Wenjing Han, Bo Liu, Tongliang The Department of Intelligent Data Science College of Computer Science and Technology National University of Defense Technology Changsha410073 China The Trustworthy Machine Learning Lab School of Computer Science Faculty of Engineering University of Sydney DarlingtonNSW2008 Australia The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China The Department of Automation University of Science and Technology of China Hefei230026 China The Department of Computer Science Hong Kong Baptist University Hong Kong

Given data with noisy labels, over-parameterized deep networks suffer overfitting mislabeled data, resulting in poor generalization. The memorization effect of deep networks shows that although the networks have the ability to memorize all noisy data, they would first memorize clean training data, and then gradually memorize mislabeled training data. A simple and effective method that exploits the memorization effect to combat noisy labels is early stopping. However, early stopping cannot distinguish the memorization of clean data and mislabeled data, resulting in the network still inevitably overfitting mislabeled data in the early training stage. In this paper, to decouple the memorization of clean data and mislabeled data, and further reduce the side effect of mislabeled data, we perform additive decomposition on network parameters. Namely, all parameters are additively decomposed into two groups, i.e. , parameters w are decomposed as w = σ + γ. Afterward, the parameters σ are considered to memorize clean data, while the parameters γ are considered to memorize mislabeled data. Benefiting from the memorization effect, the updates of the parameters σ are encouraged to fully memorize clean data in early training, and then discouraged with the increase of training epochs to reduce interference of mislabeled data. The updates of the parameters γ are the opposite. In testing, only the parameters σ are employed to enhance generalization. Extensive experiments on both simulated and real-world benchmarks confirm the superior performance of our method. The implementation is available at: https://***/wangjingyi9924/TNLPAD. Copyright © 2024, The Authors. All rights reserved.

关键词： Benchmarking

Distribution-Aware Calibration for Object Detection with Noisy Bounding Boxes

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Donghao Li, Jialin Li, Jinpeng Huang, Jiancheng Nie, Qiang Liu, Yong Gao, Bin-Bin Wang, Qiong Heng, Pheng-Ann Chen, Guangyong The Chinese University of Hong Kong Hong Kong Tencent YouTu Lab Shenzhen China Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China Guangzhou China Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China Zhejiang Lab Hangzhou China

Large-scale well-annotated datasets are of great importance for training an effective object detector. However, obtaining accurate bounding box annotations is laborious and demanding. Unfortunately, the resultant noisy bounding boxes could cause corrupt supervision signals and thus diminish detection performance. Motivated by the observation that the real ground-truth is usually situated in the aggregation region of the proposals assigned to a noisy ground-truth, we propose DIStribution-aware CalibratiOn (DISCO) to model the spatial distribution of proposals for calibrating supervision signals. In DISCO, spatial distribution modeling is performed to statistically extract the potential locations of objects. Based on the modeled distribution, three distribution-aware techniques, i.e., distribution-aware proposal augmentation (DA-Aug), distribution-aware box refinement (DA-Ref), and distribution-aware confidence estimation (DA-Est), are developed to improve classification, localization, and interpretability, respectively. Extensive experiments demonstrate that DISCO can achieve SOTA performance in this task, especially at high noise levels. Code is available at https://***/Correr-Zhou/DISCO. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

RepMode: Learning to Re-parameterize Diverse Experts for Subcellular Structure Prediction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhou, Donghao Gu, Chunbin Xu, Junde Liu, Furui Wang, Qiong Chen, Guangyong Heng, Pheng-Ann Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China The Chinese University of Hong Kong Hong Kong Zhejiang Lab China

In biological research, fluorescence staining is a key technique to reveal the locations and morphology of subcellular structures. However, it is slow, expensive, and harmful to cells. In this paper, we model it as a deep learning task termed subcellular structure prediction (SSP), aiming to predict the 3D fluorescent images of multiple subcellular structures from a 3D transmitted-light image. Unfortunately, due to the limitations of current biotechnology, each image is partially labeled in SSP. Besides, naturally, subcellular structures vary considerably in size, which causes the multi-scale issue of SSP. To overcome these challenges, we propose Re-parameterizing Mixture-of-Diverse-Experts (RepMode), a network that dynamically organizes its parameters with task-aware priors to handle specified single-label prediction tasks. In RepMode, the Mixture-of-Diverse-Experts (MoDE) block is designed to learn the generalized parameters for all tasks, and gating re-parameterization (GatRep) is performed to generate the specialized parameters for each task, by which RepMode can maintain a compact practical topology exactly like a plain network, and meanwhile achieves a powerful theoretical topology. Comprehensive experiments show that RepMode can achieve state-of-the-art overall performance in SSP. Copyright © 2022, The Authors. All rights reserved.

关键词： Forecasting

IFAST: Weakly Supervised Interpretable Face Anti-spoofing from Single-shot Binocular NIR Images

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Jiancheng Zhou, Donghao Chen, Shifeng ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen518055 China University of Chinese Academy of Sciences Beijing China Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen518055 China The Chinese University of Hong Kong Hong Kong

Single-shot face anti-spoofing (FAS) is a key technique for securing face recognition systems, and it requires only static images as input. However, single-shot FAS remains a challenging and under-explored problem due to two main reasons: 1) on the data side, learning FAS from RGB images is largely context-dependent, and single-shot images without additional annotations contain limited semantic information. 2) on the model side, existing single-shot FAS models are infeasible to provide proper evidence for their decisions, and FAS methods based on depth estimation require expensive per-pixel annotations. To address these issues, a large binocular NIR image dataset (BNI-FAS) is constructed and published, which contains more than 300,000 real face and plane attack images, and an Interpretable FAS Transformer (IFAST) is proposed that requires only weak supervision to produce interpretable predictions. Our IFAST can produce pixel-wise disparity maps by the proposed disparity estimation Transformer with Dynamic Matching Attention (DMA) block. Besides, a well-designed confidence map generator is adopted to cooperate with the proposed dual-teacher distillation module to obtain the final discriminant results. The comprehensive experiments show that our IFAST can achieve state-of-the-art results on BNI-FAS, proving the effectiveness of the single-shot FAS based on binocular NIR images. Copyright © 2023, The Authors. All rights reserved.

关键词： Distillation

Masked Image Training for Generalizable Deep Image Denoising

学校读者我要写书评

暂无评论

Masked Image Training for Generalizable Deep Image Denoising

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Haoyu Chen Jinjin Gu Yihao Liu Salma Abdel Magid Chao Dong Qiong Wang Hanspeter Pfister Lei Zhu The Hong Kong University of Science and Technology (Guangzhou) Shanghai AI Lab The University of Sydney ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences University of Chinese Academy of Sciences Harvard University Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences The Hong Kong University of Science and Technology

When capturing and storing images, devices inevitably introduce noise. Reducing this noise is a critical task called image denoising. Deep learning has become the de facto method for image denoising, especially with the emergence of Transformer-based models that have achieved notable state-of-the-art results on various image tasks. However, deep learning-based methods often suffer from a lack of generalization ability. For example, deep models trained on Gaussian noise may perform poorly when tested on other noise distributions. To address this issue, we present a novel approach to enhance the generalization performance of denoising networks, known as masked training. Our method involves masking random pixels of the input image and reconstructing the missing information during training. We also mask out the features in the self-attention layers to avoid the impact of training-testing inconsistency. Our approach exhibits better generalization ability than other deep learning models and is directly applicable to real-world scenarios. Additionally, our interpretability analysis demonstrates the superiority of our method.

关键词：

Pseudo-label Guided Cross-video Pixel Contrast for Robotic Surgical Scene Segmentation with Limited Annotations

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yu, Yang Zhao, Zixu Jin, Yueming Chen, Guangyong Dou, Qi Heng, Pheng-Ann The Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen China The Department of Computer Science University College London United Kingdom Zhejiang Lab China

Surgical scene segmentation is fundamentally crucial for prompting cognitive assistance in robotic surgery. However, pixel-wise annotating surgical video in a frame-by-frame manner is expensive and time consuming. To greatly reduce the labeling burden, in this work, we study semi-supervised scene segmentation from robotic surgical video, which is practically essential yet rarely explored before. We consider a clinically suitable annotation situation under the equidistant sampling. We then propose PGV-CL, a novel pseudo-label guided cross-video contrast learning method to boost scene segmentation. It effectively leverages unlabeled data for a trusty and global model regularization that produces more discriminative feature representation. Concretely, for trusty representation learning, we propose to incorporate pseudo labels to instruct the pair selection, obtaining more reliable representation pairs for pixel contrast. Moreover, we expand the representation learning space from previous image-level to cross-video, which can capture the global semantics to benefit the learning process. We extensively evaluate our method on a public robotic surgery dataset EndoVis18 and a public cataract dataset CaDIS. Experimental results demonstrate the effectiveness of our method, consistently outperforming the state-of-the-art semi-supervised methods under different labeling ratios, and even surpassing fully supervised training on EndoVis18 with 10.1% labeling. Our code is available at https://***/yangyu-cuhk/PGVCL. Copyright © 2022, The Authors. All rights reserved.

关键词： Robotic surgery