检索结果-内蒙古大学图书馆

Detect Any Shadow: Segment Anything for Video Shadow Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Yonghui Zhou, Wengang Mao, Yunyao Li, Houqiang The CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei230027 China Institute of Artificial Intelligence Hefei Comprehensive National Science Center China

Segment anything model (SAM) has achieved great success in the field of natural image segmentation. Nevertheless, SAM tends to consider shadows as background and therefore does not perform segmentation on them. In this paper, we propose ShadowSAM, a simple yet effective framework for fine-tuning SAM to detect shadows. Besides, by combining it with long short-term attention mechanism, we extend its capability for efficient video shadow detection. Specifically, we first fine-tune SAM on ViSha training dataset by utilizing the bounding boxes obtained from the ground truth shadow mask. Then during the inference stage, we simulate user interaction by providing bounding boxes to detect a specific frame (e.g., the first frame). Subsequently, using the detected shadow mask as a prior, we employ a long short-term network to learn spatial correlations between distant frames and temporal consistency between adjacent frames, thereby achieving precise shadow information propagation across video frames. Extensive experimental results demonstrate the effectiveness of our method, with notable margin over the state-of-the-art approaches in terms of MAE and IoU metrics. Moreover, our method exhibits accelerated inference speed compared to previous video shadow detection approaches, validating the effectiveness and efficiency of our method. The source code is now publicly available at https://***/harrytea/Detect-AnyShadow. Copyright © 2023, The Authors. All rights reserved.

关键词： Image segmentation

From multi-view to hollow-3D: Hallucinated hollow-3D R-CNN for 3D object detection

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Deng, Jiajun Zhou, Wengang Zhang, Yanyong Li, Houqiang The CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei230026 China The Department of Computer Science University of Science and Technology of China Hefei230026 China

As an emerging data modal with precise distance sensing, LiDAR point clouds have been placed great expectations on 3D scene understanding. However, point clouds are always sparsely distributed in the 3D space, and with unstructured storage, which makes it difficult to represent them for effective 3D object detection. To this end, in this work, we regard point clouds as hollow-3D data and propose a new architecture, namely Hallucinated Hollow-3D R-CNN (H23D R-CNN), to address the problem of 3D object detection. In our approach, we first extract the multi-view features by sequentially projecting the point clouds into the perspective view and the bird-eye view. Then, we hallucinate the 3D representation by a novel bilaterally guided multi-view fusion block. Finally, the 3D objects are detected via a box refinement module with a novel Hierarchical Voxel RoI Pooling operation. The proposed H23D R-CNN provides a new angle to take full advantage of complementary information in the perspective view and the bird-eye view with an efficient framework. We evaluate our approach on the public KITTI Dataset and Waymo Open Dataset. Extensive experiments demonstrate the superiority of our method over the state-of-the-art algorithms with respect to both effectiveness and efficiency. The code will be made available at https://***/djiajunustc/H-23D_R-CNN. Copyright © 2021, The Authors. All rights reserved.

关键词： Object recognition

AdaptVision: Dynamic Input Scaling in MLLMs for Versatile Scene Understanding

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Yonghui Zhou, Wengang Feng, Hao Li, Houqiang The CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei230027 China Institute of Artificial Intelligence Hefei Comprehensive National Science Center China

Over the past few years, the advancement of Multimodal Large Language Models (MLLMs) has captured the wide interest of researchers, leading to numerous innovations to enhance MLLMs’ comprehension. In this paper, we present AdaptVision, a multimodal large language model specifically designed to dynamically process input images at varying resolutions. We hypothesize that the requisite number of visual tokens for the model is contingent upon both the resolution and content of the input image. Generally, natural images with a lower information density can be effectively interpreted by the model using fewer visual tokens at reduced resolutions. In contrast, images containing textual content, such as documents with rich text, necessitate a higher number of visual tokens for accurate text interpretation due to their higher information density. Building on this insight, we devise a dynamic image partitioning module that adjusts the number of visual tokens according to the size and aspect ratio of images. This method mitigates distortion effects that arise from resizing images to a uniform resolution and dynamically optimizing the visual tokens input to the LLMs. Our model is capable of processing images with resolutions up to 1008 × 1008. Extensive experiments across various datasets demonstrate that our method achieves impressive performance in handling vision-language tasks in both natural and text-related scenes. The source code and dataset are now publicly available at https://***/harrytea/AdaptVision. Copyright © 2024, The Authors. All rights reserved.

关键词： Modeling languages

Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Hu, Zexi Chen, Xiaoming Wing, Henry Yeung, Fung Chung, Yuk Ying Chen, Zhibo School of Computer Science University of Sydney Australia School of Computer Science and Engineering Beijing Technology and Business University China CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System University of Science and Technology of China China

Despite the recent progress in light field super-resolution (LFSR) achieved by convolutional neural networks, the correlation information of light field (LF) images has not been sufficiently studied and exploited due to the complexity of 4D LF data. To cope with such high-dimensional LF data, most of the existing LFSR methods resorted to decomposing it into lower dimensions and subsequently performing optimization on the decomposed sub-spaces. However, these methods are inherently limited as they neglected the characteristics of the decomposition operations and only utilized a limited set of LF sub-spaces ending up failing to sufficiently extract spatio-angular features and leading to a performance bottleneck. To overcome these limitations, in this paper, we comprehensively discover the potentials of LF decomposition and propose a novel concept of decomposition kernels. In particular, we systematically unify the decomposition operations of various sub-spaces into a series of such decomposition kernels, which are incorporated into our proposed Decomposition Kernel Network (DKNet) for comprehensive spatio-angular feature extraction. The proposed DKNet is experimentally verified to achieve considerable improvements compared with the state-of-the-art methods. To further improve DKNet in producing more visually pleasing LFSR results, based on the VGG network, we propose a LFVGG loss to guide the Texture-Enhanced DKNet (TE-DKNet) to generate rich authentic textures and enhance LF images’ visual quality significantly. We also propose an indirect evaluation metric by taking advantage of LF material recognition to objectively assess the perceptual enhancement brought by the LFVGG loss. Copyright © 2021, The Authors. All rights reserved.

关键词： Optical resolving power

Beyond triplet loss: Meta prototypical N-tuple loss for person re-identification

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhang, Zhizheng Lan, Cuiling Zeng, Wenjun Chen, Zhibo Chang, Shih-Fu The CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Anhui Hefei China Microsoft Research Asia Beijing China Columbia University New YorkNY United States

Person Re-identification (ReID) aims at matching a person of interest across images. In convolutional neural network (CNN) based approaches, loss design plays a vital role in pulling closer features of the same identity and pushing far apart features of different identities. In recent years, triplet loss achieves superior performance and is predominant in ReID. However, triplet loss considers only three instances of two classes in per-query optimization (with an anchor sample as query) and it is actually equivalent to a two-class classification. There is a lack of loss design which enables the joint optimization of multiple instances (of multiple classes) within per-query optimization for person ReID. In this paper, we introduce a multi-class classification loss, i.e., N-tuple loss, to jointly consider multiple (N) instances for per-query optimization. This in fact aligns better with the ReID test/inference process, which conducts the ranking/comparisons among multiple instances. Furthermore, for more efficient multi-class classification, we propose a new meta prototypical N-tuple loss. With the multi-class classification incorporated, our model achieves the state-of-the-art performance on the benchmark person ReID datasets. Copyright © 2020, The Authors. All rights reserved.

关键词： Classification (of information)

Progressive unsupervised person re-identification by tracklet association with spatio-temporal regularization

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Xie, Qiaokang Zhou, Wengang Qi, Guo-Jun Tian, Qi Li, Houqiang CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei230027 China Huawei Cloud EI Product Department Huawei Noah's Ark Laboratory

Existing methods for person re-identification (Re-ID) are mostly based on supervised learning which requires numerous manually labeled samples across all camera views for training. Such a paradigm suffers the scalability issue since in real-world Re-ID application, it is difficult to exhaustively label abundant identities over multiple disjoint camera views. To this end, we propose a progressive deep learning method for unsupervised person Re-ID in the wild by Tracklet Association with Spatio-Temporal Regularization (TASTR). In our approach, we first collect tracklet data within each camera by automatic person detection and tracking. Then, an initial Re-ID model is trained based on within-camera triplet construction for person representation learning. After that, based on the person visual feature and spatio-temporal constraint, we associate cross-camera tracklets to generate cross-camera triplets and update the Re-ID model. Lastly, with the refined Re-ID model, better visual feature of person can be extracted, which further promote the association of cross-camera tracklets. The last two steps are iterated multiple times to progressively upgrade the Re-ID model. To facilitate the study, we have collected a new 4K UHD video dataset named Campus-4K with full frames and full spatio-temporal information. Experimental results show that with the spatio-temporal constraint in the training phase, the proposed approach outperforms the state-of-the-art unsupervised methods by notable margins on DukeMTMC-reID, and achieves competitive performance to fully supervised methods on both DukeMTMC-reID and Campus-4K datasets. Copyright © 2019, The Authors. All rights reserved.

关键词： Cameras

DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhao, Youpeng Zhao, Jian Hu, Xunhan Zhou, Wengang Li, Houqiang CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei230027 China Institute of Artificial Intelligence Hefei Comprehensive National Science Center Hefei China

Recent years have witnessed the great breakthrough of deep reinforcement learning (DRL) in various perfect and imperfect information games. Among these games, DouDizhu, a popular card game in China, is very challenging due to the imperfect information, large state space, elements of collaboration and a massive number of possible moves from turn to turn. Recently, a DouDizhu AI system called DouZero has been proposed. Trained using traditional Monte Carlo method with deep neural networks and self-play procedure without the abstraction of human prior knowledge, DouZero has outperformed all the existing DouDizhu AI programs. In this work, we propose to enhance DouZero by introducing opponent modeling into DouZero. Besides, we propose a novel coach network to further boost the performance of DouZero and accelerate its training process. With the integration of the above two techniques into DouZero, our DouDizhu AI system achieves better performance and ranks top in the Botzone leaderboard among more than 400 AI agents, including DouZero. Copyright © 2022, The Authors. All rights reserved.

关键词： Reinforcement learning

Transform-Invariant Convolutional Neural Networks for Image Classification and Search

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Shen, Xu Tian, Xinmei He, Anfeng Sun, Shaoyan Tao, Dacheng CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System University of Science and Technology of China Hefei Anhui230027 China Centre for Quantum Computation & Intelligent Systems Faculty of Engineering and Information Technology University of Technology Sydney UltimoNSW2007 Australia

Convolutional neural networks (CNNs) have achieved state-of-the-art results on many visual recognition tasks. However, current CNN models still exhibit a poor ability to be invariant to spatial transformations of images. Intuitively, with sufficient layers and parameters, hierarchical combinations of convolution (matrix multiplication and nonlinear activation) and pooling operations should be able to learn a robust mapping from transformed input images to transform-invariant representations. In this paper, we propose randomly transforming (rotation, scale, and translation) feature maps of CNNs during the training stage. This prevents complex dependencies of specific rotation, scale, and translation levels of training images in CNN models. Rather, each convolutional kernel learns to detect a feature that is generally helpful for producing the transform-invariant answer given the combinatorially large variety of transform levels of its input feature maps. In this way, we do not require any extra training supervision or modification to the optimization process and training images. We show that random transformation provides significant improvements of CNNs on many benchmark tasks, including small-scale image recognition, large-scale image recognition, and image retrieval. The code is available at https: ACM ISBN 978-1-4503-2138-9. Copyright © 2019, The Authors. All rights reserved.

关键词： Image recognition

Real-time correlation tracking via joint model compression and transfer

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wang, Ning Zhou, Wengang Song, Yibing Ma, Chao Li, Houqiang CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System Department of Electronic Engineering and Information Science University of Science and Technology of China Hefei China Tencent AI Lab Shenzhen China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

Correlation filters (CF) have received considerable attention in visual tracking because of their computational efficiency. Leveraging deep features via off-the-shelf CNN models (e.g., VGG), CF trackers achieve state-of-the-art performance while consuming a large number of computing resources. This limits deep CF trackers to be deployed to many mobile platforms on which only a single-core CPU is available. In this paper, we propose to jointly compress and transfer off-the-shelf CNN models within a knowledge distillation framework. We formulate a CNN model pretrained from the image classification task as a teacher network, and distill this teacher network into a lightweight student network as the feature extractor to speed up CF trackers. In the distillation process, we propose a fidelity loss to enable the student network to maintain the representation capability of the teacher network. Meanwhile, we design a tracking loss to adapt the objective of the student network from object recognition to visual tracking. The distillation process is performed offline on multiple layers and adaptively updates the student network using a background-aware online learning scheme. Extensive experiments on five challenging datasets demonstrate that the lightweight student network accelerates the speed of state-of-the-art deep CF trackers to real-time on a single-core CPU while maintaining almost the same tracking accuracy. Copyright © 2019, The Authors. All rights reserved.

关键词： Object recognition