检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Shang, Zirui Zhu, Yubo Li, Hongxi Yang, Shuo Wu, Xinxiao Beijing Key Laboratory of Intelligent Information Technology School of Computer Science & Technology Beijing Institute of Technology China Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China

Video summarization aims to eliminate visual redundancy while retaining key parts of video to construct concise and comprehensive synopses. Most existing methods use discriminative models to predict the importance scores of video frames. However, these methods are susceptible to annotation inconsistency caused by the inherent subjectivity of different annotators when annotating the same video. In this paper, we introduce a generative framework for video summarization that learns how to generate summaries from a probability distribution perspective, effectively reducing the interference of subjective annotation noise. Specifically, we propose a novel diffusion summarization method based on the Denoising Diffusion Probabilistic Model (DDPM), which learns the probability distribution of training data through noise prediction, and generates summaries by iterative denoising. Our method is more resistant to subjective annotation noise, and is less prone to overfitting the training data than discriminative methods, with strong generalization ability. Moreover, to facilitate training DDPM with limited data, we employ an unsupervised video summarization model to implement the earlier denoising process. Extensive experiments on various datasets (TVSum, SumMe, and FPVSum) demonstrate the effectiveness of our method. Copyright © 2024, The Authors. All rights reserved.

关键词： Image denoising

来源：评论

学校读者我要写书评

暂无评论

M2Trans: Multi-Modal Regularized Coarse-to-Fine Transformer for Ultrasound Image Super-Resolution

引用

IEEE Journal of Biomedical and Health Informatics 2024年第5期29卷 3112-3123页

作者： Ni, Zhangkai Xiao, Runyu Yang, Wenhan Wang, Hanli Wang, Zhihua Xiang, Lihua Sun, Liping Tongji University Department of Computer Science and Technology The Key Laboratory of Embedded System and Service Computing Ministry of Education Shanghai200092 China Pengcheng Laboratory Shenzhen518066 China Shenzhen MSU-BIT University Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen518115 China Tongji University Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment Department of Medical Ultrasound Center of Tumor Minimally Invasive Treatment Shanghai Tenth Peoples Hospital Ultrasound Research and Education Institute Clinical Research Center for Interventional Medicine School of Medicine Shanghai200072 China

Ultrasound image super-resolution (SR) aims to transform low-resolution images into high-resolution ones, thereby restoring intricate details crucial for improved diagnostic accuracy. However, prevailing methods relying solely on image modality guidance and pixel-wise loss functions struggle to capture the distinct characteristics of medical images, such as unique texture patterns and specific colors harboring critical diagnostic information. To overcome these challenges, this paper introduces the Multi-Modal Regularized Coarse-to-fine Transformer (M2Trans) for Ultrasound Image SR. By integrating the text modality, we establish joint image-text guidance during training, leveraging the medical CLIP model to incorporate richer priors from text descriptions into the SR optimization process, enhancing detail, structure, and semantic recovery. Furthermore, we propose a novel coarse-to-fine transformer comprising multiple branches infused with self-attention and frequency transforms to efficiently capture signal dependencies across different scales. Extensive experimental results demonstrate significant improvements over state-of-the-art methods on benchmark datasets, including CCA-US, US-CASE, and our newly created dataset MMUS1K, with a minimum improvement of 0.17dB, 0.30dB, and 0.28dB in terms of PSNR. Our code and dataset will be available at: https://***/eezkni/M2Trans © 2013 IEEE.

关键词： Ultrasonic imaging

来源：评论

学校读者我要写书评

暂无评论

Human reappearance detection based on on-line learning

Human reappearance detection based on on-line learning

引用

作者： Hu, Lei Wang, Yizhou Jiang, Shuqiang Huang, Qingming Gao, Wen Key Lab of Intelligent Information Processing Chinese Academy of Sciences Beijing 100080 China Institute of Computing Technology Chinese Academy of Sciences Beijing 100080 China Graduate University Chinese Academy of Sciences Beijing 100049 China Institute of Digital Media Peking University Beijing 100871 China Key Lab of Machine Perception Peking University MOE China

ISBN: (纸本)9781424421756

Many video surveillance applications require detecting human reappearances in a scene monitored by a camera or over a network of cameras. This is the human reappearance detection (HRD) problem. Studying this problem is important for analyzing a surveillance scenario at semantic level. In this paper, we propose a novel online learning frameworkfor solving HRD problem. Both generative model and discriminative model are employed in this framework and a voting scheme is presented to fuse the decisions ofboth models for determining whether a just entered person is one of those who have shown up, i.e. whether a reappearance happens. Both models will be updated based on mistake-driven online learning strategy. Our experimental results show that the adopted online learning framework not only improves the reappearance detection accuracy but also achieves high robustness in various surveillance scenes. © 2008 IEEE.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

Temporally Consistent Stereo Matching

arXiv

引用

arXiv 2024年

作者： Zeng, Jiaxi Yao, Chengtang Wu, Yuwei Jia, Yunde Beijing Key Laboratory of Intelligent Information Technology School of Computer Science & Technology Beijing Institute of Technology China Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China Horizon Robotics China

Stereo matching provides depth estimation from binocular images for downstream applications. These applications mostly take video streams as input and require temporally consistent depth maps. However, existing methods mainly focus on the estimation at the single-frame level. This commonly leads to temporally inconsistent results, especially in ill-posed regions. In this paper, we aim to leverage temporal information to improve the temporal consistency, accuracy, and efficiency of stereo matching. To achieve this, we formulate video stereo matching as a process of temporal disparity completion followed by continuous iterative refinements. Specifically, we first project the disparity of the previous timestamp to the current viewpoint, obtaining a semi-dense disparity map. Then, we complete this map through a disparity completion module to obtain a well-initialized disparity map. The state features from the current completion module and from the past refinement are fused together, providing a temporally coherent state for subsequent refinement. Based on this coherent state, we introduce a dual-space refinement module to iteratively refine the initialized result in both disparity and disparity gradient spaces, improving estimations in ill-posed regions. Extensive experiments demonstrate that our method effectively alleviates temporal inconsistency while enhancing both accuracy and efficiency. Currently, our method ranks second on the KITTI 2015 benchmark, while achieving superior efficiency compared to other state-of-the-art methods. The code is available at https://***/jiaxiZeng/Temporally-Consistent-Stereo-Matching. © 2024, CC BY-NC-ND.

关键词： Efficiency

来源：评论

学校读者我要写书评

暂无评论

Block-Diagonal Guided DBSCAN Clustering

arXiv

引用

arXiv 2024年

作者： Xing, Zheng Zhao, Weibing Future Network of Intelligence Institute School of Science and Engineering The Chinese University of Hong Kong Shenzhen China Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China

Cluster analysis plays a crucial role in database mining, and one of the most widely used algorithms in this field is DBSCAN. However, DBSCAN has several limitations, such as difficulty in handling high-dimensional large-scale data, sensitivity to input parameters, and lack of robustness in producing clustering results. This paper introduces an improved version of DBSCAN that leverages the block-diagonal property of the similarity graph to guide the clustering procedure of DBSCAN. The key idea is to construct a graph that measures the similarity between high-dimensional large-scale data points and has the potential to be transformed into a block-diagonal form through an unknown permutation, followed by a cluster-ordering procedure to generate the desired permutation. The clustering structure can be easily determined by identifying the diagonal blocks in the permuted graph. Thus, the main challenge is to construct a graph with a potential block-diagonal form, permute the graph to achieve a block-diagonal structure, and automatically identify diagonal blocks in the permuted graph. To tackle these challenges, we first formulate a block-diagonal constrained self-representation problem to construct a similarity graph of high-dimensional points with a potential block-diagonal form after an unknown permutation. We propose a gradient descent-based method to solve the proposed problem. Additionally, we develop a DBSCAN-based points traversal algorithm that identifies clusters with high densities in the graph and generates an augmented ordering of clusters. The block-diagonal structure of the graph is then achieved through permutation based on the traversal order, providing a flexible foundation for both automatic and interactive cluster analysis. We introduce a split-and-refine algorithm to automatically search for all diagonal blocks in the permuted graph with theoretically optimal guarantees under specific cases. We extensively evaluate our proposed approach on twelve chal

关键词： Cluster analysis

来源：评论

学校读者我要写书评

暂无评论

FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models 38

FIRE: A Dataset for Feedback Integration and Refinement Eval...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Li, Pengxiang Gao, Zhi Zhang, Bofei Yuan, Tao Wu, Yuwei Harandi, Mehrtash Jia, Yunde Zhu, Song-Chun Li, Qing Beijing Key Laboratory of Intelligent Information Technology School of Computer Science & Technology Beijing Institute of Technology China State Key Laboratory of General Artificial Intelligence BIGAI China State Key Laboratory of General Artificial Intelligence Peking University China Department of Electrical and Computer System Engineering Monash University Australia Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China Department of Automation Tsinghua University China

Vision language models (VLMs) have achieved impressive progress in diverse applications, becoming a prevalent research direction. In this paper, we build FIRE, a feedback-refinement dataset, consisting of 1.1M multi-turn conversations that are derived from 27 source datasets, empowering VLMs to spontaneously refine their responses based on user feedback across diverse tasks. To scale up the data collection, FIRE is collected in two components: FIRE-100K and FIRE-1M, where FIRE-100K is generated by GPT-4V, and FIRE-1M is freely generated via models trained on FIRE-100K. Then, we build FIRE-Bench, a benchmark to comprehensively evaluate the feedback-refining capability of VLMs, which contains 11K feedback-refinement conversations as the test data, two evaluation settings, and a model to provide feedback for VLMs. We develop the FIRE-LLaVA model by fine-tuning LLaVA on FIRE-100K and FIRE-1M, which shows remarkable feedback-refining capability on FIRE-Bench and outperforms untrained VLMs by 50%, making more efficient user-agent interactions and underscoring the significance of the FIRE dataset. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Hyperbolic feature augmentation via distribution estimation and infinite sampling on manifolds 22

Hyperbolic feature augmentation via distribution estimation ...

引用

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Zhi Gao Yuwei Wu Yunde Jia Mehrtash Harandi Beijing Lab of Intellegent Information Technology School of Computer Science Beijing Institute of Technology China Beijing Lab of Intellegent Information Technology School of Computer Science Beijing Institute of Technology China and Guangdong Lab of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China Guangdong Lab of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China and Beijing Lab of Intellegent Information Technology School of Computer Science Beijing Institute of Technology China Department of Electrical and Computer Systems Eng. Monash University and Data61 Australia

ISBN: (纸本)9781713871088

Learning in hyperbolic spaces has attracted growing attention recently, owing to their capabilities in capturing hierarchical structures of data. However, existing learning algorithms in the hyperbolic space tend to overfit when limited data is given. In this paper, we propose a hyperbolic feature augmentation method that generates diverse and discriminative features in the hyperbolic space to combat overfitting. We employ a wrapped hyperbolic normal distribution to model augmented features, and use a neural ordinary differential equation module that benefits from meta-learning to estimate the distribution. This is to reduce the bias of estimation caused by the scarcity of data. We also derive an upper bound of the augmentation loss, which enables us to train a hyperbolic model by using an infinite number of augmentations. Experiments on few-shot learning and continual learning tasks show that our method significantly improves the performance of hyperbolic algorithms in scarce data regimes.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Sparse Point Guided 3D Lane Detection

Sparse Point Guided 3D Lane Detection

引用

International Conference on Computer Vision (ICCV)

作者： Chengtang Yao Lidong Yu Yuwei Wu Yunde Jia School of Computer Science & Technology Beijing Key Laboratory of Intelligent Information Technology Beijing Institute of Technology China Autonomous Driving Algorithm DeepRoute Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China

3D lane detection usually builds a dense correspondence between the front-view space and the BEV space to estimate lane points in the 3D space. 3D lanes only occupy a small ratio of the dense correspondence, while most correspondence belongs to the redundant background. This sparsity phenomenon bottlenecks valuable computation and raises the computation cost of building a high-resolution correspondence for accurate results. In this paper, we propose a sparse point-guided 3D lane detection, focusing on points related to 3D lanes. Our method runs in a coarse-to-fine manner, including coarse-level lane detection and iterative fine-level sparse point refinements. In coarse-level lane detection, we build a dense but efficient correspondence between the front view and BEV space at a very low resolution to compute coarse lanes. Then in fine-level sparse point refinement, we sample sparse points around coarse lanes to extract local features from the high-resolution front-view feature map. The high-resolution local information brought by sparse points refines 3D lanes in the BEV space hierarchically from low resolution to high resolution. The sparse point guides a more effective information flow and greatly promotes the SOTA result by 3 points on the overall F1-score and 6 points on several hard situations while reducing almost half memory cost and speeding up 2 times.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understanding

arXiv

引用

arXiv 2023年

作者： Zhai, Mingliang Li, Yulin Qin, Xiameng Yi, Chen Xie, Qunyi Zhang, Chengquan Yao, Kun Wu, Yuwei Jia, Yunde Beijing Key Laboratory of Intelligent Information Technology School of Computer Science & Technology Beijing Institute of Technology China Baidu Inc. China Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China

Transformers achieve promising performance in document understanding because of their high effectiveness and still suffer from quadratic computational complexity dependency on the sequence length. General efficient transformers are challenging to be directly adapted to model document. They are unable to handle the layout representation in documents, e.g. word, line and paragraph, on different granularity levels and seem hard to achieve a good trade-off between efficiency and performance. To tackle the concerns, we propose Fast-StrucTexT, an efficient multi-modal framework based on the StrucTexT algorithm with an hourglass transformer architecture, for visual document understanding. Specifically, we design a modality-guided dynamic token merging block to make the model learn multi-granularity representation and prunes redundant tokens. Additionally, we present a multi-modal interaction module called Symmetry Cross Attention (SCA) to consider multi-modal fusion and efficiently guide the token mergence. The SCA allows one modality input as query to calculate cross attention with another modality in a dual phase. Extensive experiments on FUNSD, SROIE, and CORD datasets demonstrate that our model achieves the state-of-the-art performance and almost 1.9× faster inference time than the state-of-the-art methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Economic and social effects

来源：评论

学校读者我要写书评

暂无评论

Parameterized Cost Volume for Stereo Matching

Parameterized Cost Volume for Stereo Matching

引用

International Conference on Computer Vision (ICCV)

作者： Jiaxi Zeng Chengtang Yao Lidong Yu Yuwei Wu Yunde Jia Beijing Key Laboratory of Intelligent Information Technology School of Computer Science & Technology Beijing Institute of Technology China Autonomous Driving Algorithm Deeproute Guangdong Laboratory of Machine Perception and Intelligent Computing Shenzhen MSU-BIT University China

Stereo matching becomes computationally challenging when dealing with a large disparity range. Prior methods mainly alleviate the computation through dynamic cost volume by focusing on a local disparity space, but it requires many iterations to get close to the ground truth due to the lack of a global view. We find that the dynamic cost volume approximately encodes the disparity space as a single Gaussian distribution with a fixed and small variance at each iteration, which results in an inadequate global view over disparity space and a small update step at every iteration. In this paper, we propose a parameterized cost volume to encode the entire disparity space using multi-Gaussian distribution. The disparity distribution of each pixel is parameterized by weights, means, and variances. The means and variances are used to sample disparity candidates for cost computation, while the weights and means are used to calculate the disparity output. The above parameters are computed through a JS-divergence-based optimization, which is realized as a gradient descent update in a feed-forward differential module. Experiments show that our method speeds up the runtime of RAFT-Stereo by 4 ~ 15 times, achieving real-time performance and comparable accuracy. The code is available at https://***/jiaxiZeng/Parameterized-Cost-Volume-for-Stereo-Matching.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：