检索结果-内蒙古大学图书馆

Temporo-Spatial Parallel Sparse Memory Networks for Efficient video object segmentation

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2024年第11期25卷 17291-17304页

作者： Dang, Jisheng Zheng, Huicheng Wang, Bimei Wang, Longguang Guo, Yulan Sun Yat Sen Univ Sch Comp Sci & Engn Guangzhou 510006 Peoples R China Minist Educ Key Lab Machine Intelligence & Adv Comp Guangzhou 510006 Peoples R China Guangdong Key Lab Informat Secur Technol Guangzhou 510006 Peoples R China Jinan Univ Coll Informat Sci & Technol Guangzhou 510006 Peoples R China Aviat Univ Air Force Coll Elect Sci & Technol Changchun 130022 Peoples R China Sun Yat Sen Univ Sch Elect & Commun Engn Shenzhen 518107 Peoples R China

Memory-based networks have achieved tremendous success in video object segmentation. However, these methods still suffer from unfaithful segmentation and inferior efficiency under complicated video scenarios. The reasons are mainly threefold: 1) Weak perception of fast-moving targets due to individual frame memory patterns without capturing inter-frame motion;2) Lack of discrimination to visually similar appearances due to the limited receptive field;3) Redundant computation caused by matching with all memorized frames. To address these issues, we propose a Temporo-Spatial Parallel Sparse Memory network (TSPSM) for efficient video object segmentation. Our TSPSM constructs a temporal memory bank and a spatial memory bank in parallel to memorize complementary discriminative object cues. The temporal bank exploits discriminative temporal motion cues, while the spatial bank mines spatial context cues between adjacent frames with large receptive fields, thereby alleviating the ambiguity caused by similar instances and fast movements. To reduce redundant computation without sacrificing performance during the matching step, we further design a parallel sparse memory reader based on the constructed informative memory banks, which efficiently retrieves relevant temporal and spatial information in a parallel way. Experiments demonstrate that our TSPSM achieves state-of-the-art performance with real-time speed on DAVIS, and YouTube-VOS benchmarks. Furthermore, extensive experiments show that the proposed TSPMC module can be applied to existing methods as a generic plugin to significantly improve performance.

关键词： Memory management object segmentation Task analysis Motion segmentation Accuracy Visualization Decoding video object segmentation temporo-spatial parallel sparse memory network temporo-spatial parallel memory construction parallel sparse memory reader

来源：评论

学校读者我要写书评

暂无评论

A New Spatio-Temporal Saliency-Based video object segmentation (vol 8, pg 629, 2016)

引用

COGNITIVE COMPUTATION 2016年第4期8卷 648-648页

作者： Tu, Zhengzheng Abel, Andrew Zhang, Lei Luo, Bin Hussain, Amir Anhui Univ Sch Comp Sci & Technol Hefei 230601 Peoples R China Univ Stirling Comp Sci & Math Stirling FK9 4LA Scotland

Humans and animals are able to segment visual scenes by having the natural cognitive ability to quickly identify salient objects in both static and dynamic scenes. In this paper, we present a new spatio-temporal-based approach to video object segmentation that considers both motion- and image-based saliency to produce a weighted approach which can segment both static and dynamic objects. We perform fast optical flow and then calculate the motion saliency based on this temporal information, detecting the presence of global motion and adjusting the initial optical flow results accordingly. This is then fused with a region-based contrast image saliency method, with both techniques weighted. Finally, our joint weighted saliency map is used as part of a foreground-background labelling approach to produce the final segmented video files. Good results in a wide range of environments are presented, showing that our spatio-temporal system is more robust and consistent than a number of other state-of-the-art approaches.

关键词： video object segmentation Global motion Spatio-temporal saliency Foreground-background labelling

来源：评论

学校读者我要写书评

暂无评论

Integrating instance-level knowledge to see the unseen: A two-stream network for video object segmentation

引用

NEUROCOMPUTING 2024年 602卷

作者： Lu, Hannan Tian, Zhi Wei, Pengxu Ren, Haibing Zuo, Wangmeng Harbin Inst Technol Sch Comp Sci & Technol 92 Xidazhi St Harbin 150006 Peoples R China

Existing matching-based video object segmentation (VOS) approaches carry inherent limitations in segmenting pixels that have never appeared in the previous frames ( i.e. , unseen pixels). In this paper, we introduce a T wo- S tream N etwork (TSN), which addresses this issue by distinguishing between seen and unseen pixels softly and processes them with two streams. Particularly, a pixel division module is devised to generate a routing map, distinguishing between seen and unseen pixels. Guided by the routing map, TSN integrates instance-level knowledge from an instance stream and pixel-level information from a pixel stream explicitly, generating the final segmentation result. The soft partitioning strategy allows for flexibility and adaptability in the fusion process. Additionally, the compact instance stream encodes and leverages instance-level knowledge, resulting in improved segmentation accuracy of the unseen pixels. Extensive experiments demonstrate the effectiveness of our proposed TSN, and we also report state-of-the-art performance on public VOS benchmarks.

关键词： video object segmentation Matching-based Two-stream network Pixel division Instance stream

来源：评论

学校读者我要写书评

暂无评论

Adaptive ROI generation for video object segmentation using reinforcement learning

引用

PATTERN RECOGNITION 2020年 106卷 107465-107465页

作者： Sun, Mingjie Xiao, Jimin Lim, Eng Gee Xie, Yanchun Feng, Jiashi Xian Jiaotong Liverpool Univ Suzhou Peoples R China Univ Liverpool Liverpool Merseyside England Natl Univ Singapore Singapore Singapore

The task of the proposed method is semi-supervised video object segmentation where only the ground-truth segmentation of the first frame is provided. The existing approaches rely on selecting the region of interest for model update;however it is rough and inflexible, leading to performance degradation. To overcome this limitation, a novel approach is proposed which utilizes reinforcement learning to select optimal adaptation areas for each frame, based on the historical segmentation information. The RL model learns to take optimal actions to adjust the region of interest inferred from the previous frame for online model updating. To speed up the model adaption, a novel multi-branch tree based exploration method is designed to quickly select the best state action pairs. The proposed method is evaluated on three common video object segmentation datasets including DAVIS 2016, SegTrack V2 and Youtube-object. The results show that the proposed work improves the state-of-the-art of the mean region similarity to 87.1% on the DAVIS 2016 dataset, and to 79.5% on the Youtube-object dataset. Meanwhile, competitive performance is obtained on the SegTrack V2 dataset. Code is at https://***/insomnia94/ARG. (C) 2020 Elsevier Ltd. All rights reserved.

关键词： Model adaptation video object segmentation Reinforcement learning Training accelerate

来源：评论

学校读者我要写书评

暂无评论

A temporal attention based appearance model for video object segmentation

引用

APPLIED INTELLIGENCE 2022年第2期52卷 2290-2300页

作者： Wang, Hui Liu, Weibin Xing, Weiwei Beijing Jiaotong Univ Inst Informat Sci Beijing 100044 Peoples R China Beijing Jiaotong Univ Sch Software Engn Beijing 100044 Peoples R China

More and more researchers have recently paid attention to video object segmentation because it is an important building block for numerous computer vision applications. Although many algorithms promote its development, there are still some open challenges. Efficient and robust pipelines are needed to address appearance changes and the distraction from similar background objects in the video object segmentation. This paper proposes a novel neural network that integrates a temporal attention based appearance model and a boundary-aware loss. The appearance model fuses the appearance information of the first frame, the previous frame, and the current frame in the feature space, which assists the proposed method to learn a discriminative and robust target representation and avoid the drift problem of traditional propagation schemes. Moreover, the boundary-aware loss is employed for network training. Equipped with the boundary-aware loss, the proposed method achieves more accurate segmentation results with clear boundaries. The proposed method is compared with several recent state-of-the-art algorithms on popular benchmark datasets. Comprehensive experiments show that the proposed method achieves favorable performance with a high frame rate.

关键词： video object segmentation Convolutional neural networks Appearance model Mixture loss

来源：评论

学校读者我要写书评

暂无评论

Efficient and Robust video object segmentation Through Isogenous Memory Sampling and Frame Relation Mining

引用

IEEE TRANSACTIONS ON IMAGE PROCESSING 2023年 32卷 3924-3938页

作者： Dang, Jisheng Zheng, Huicheng Lai, Jinming Yan, Xu Guo, Yulan Sun Yat Sen Univ Sch Comp Sci & Engn Guangzhou 510006 Peoples R China Minist Educ Key Lab Machine Intelligence & Adv Comp Guangzhou 510006 Peoples R China GuangDong Prov Key Lab Informat Secur Technol Guangzhou 510006 Peoples R China Chinese Univ Hong Kong Future Network Intelligence Inst FNii Deep Bit Lab Shenzhen 518172 Peoples R China Chinese Univ Hong Kong Sch Sci & Engn Shenzhen 518172 Peoples R China Sun Yat Sen Univ Sch Elect & Commun Engn Shenzhen Campus Shenzhen 510275 Peoples R China

Recently, memory-based methods have achieved remarkable progress in video object segmentation. However, the segmentation performance is still limited by error accumulation and redundant memory, primarily because of 1) the semantic gap caused by similarity matching and memory reading via heterogeneous key-value encoding;2) the continuously growing and inaccurate memory through directly storing unreliable predictions of all previous frames. To address these issues, we propose an efficient, effective, and robust segmentation method based on Isogenous Memory Sampling and Frame-Relation mining (IMSFR). Specifically, by utilizing an isogenous memory sampling module, IMSFR consistently conducts memory matching and reading between sampled historical frames and the current frame in an isogenous space, minimizing the semantic gap while speeding up the model through an efficient random sampling. Furthermore, to avoid key information loss during the sampling process, we further design a frame-relation temporal memory module to mine inter-frame relations, thereby effectively preserving contextual information from the video sequence and alleviating error accumulation. Extensive experiments demonstrate the effectiveness and efficiency of the proposed IMSFR method. In particular, our IMSFR achieves state-of-the-art performance on six commonly used benchmarks in terms of region similarity & contour accuracy and speed. Our model also exhibits strong robustness against frame sampling due to its large receptive field.

关键词： video object segmentation isogenous memory sampling frame-relation mining temporal memory

来源：评论

学校读者我要写书评

暂无评论

LiDAR video object segmentation with dynamic kernel refinement

引用

PATTERN RECOGNITION LETTERS 2024年 178卷 21-27页

作者： Mei, Jianbiao Yang, Yu Wang, Mengmeng Li, Zizhang Ra, Jongwon Liu, Yong Zhejiang Univ Inst Cyber Syst & Control Hangzhou 310027 Peoples R China

In this paper, we formalize memory-and tracking-based methods to perform the LiDAR-based video object segmentation (VOS) task, which segments points of the specific 3D target (given in the first frame) in a LiDAR sequence. LiDAR-based VOS can directly provide target-aware geometric information for practical application scenarios like behavior analysis and anticipating danger. We first construct a LiDAR-based VOS dataset named KITTI-VOS based on SemanticKITTI, which acts as a testbed and facilitates comprehensive evaluations of algorithm performance. Next, we provide two types of baselines, i.e., memory-based and tracking-based baselines, to explore this task. Specifically, the first memory-based pipeline is built on a space-time memory network equipped with the non-local spatiotemporal attention-based memory bank. We further design a more potent variant to introduce the locality into the spatiotemporal attention module by local self-attention and cross-attention modules. For the second tracking-based baseline, we modify two representative 3D object tracking methods to adapt to LiDAR-based VOS tasks. Finally, we propose a refine module that takes mask priors and generates object-aware kernels, which could boost all the baselines' performance. We evaluate the proposed methods on the dataset and demonstrate their effectiveness.

关键词： LiDAR segmentation video object segmentation Dynamic kernel

来源：评论

学校读者我要写书评

暂无评论

Bilateral Temporal Re-Aggregation for Weakly-Supervised video object segmentation

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR video TECHNOLOGY 2022年第7期32卷 4498-4512页

作者： Lin, Fanchao Xie, Hongtao Liu, Chuanbin Zhang, Yongdong Univ Sci & Technol China Natl Engn Lab Brain Inspired Intelligence Technol Hefei 230026 Peoples R China

Weakly-supervised video object segmentation is an emerging video task to track and segment the target given a simple bounding box label, which requires the method to fully catch and utilize the target information. Mast existing approaches only rely on the guidance of a single frame and ignore the interaction between different frames when gathering information, making them hard to achieve reliable target representation. In this paper, we propose to capture the temporal dependencies and gather information from multiple frames through bilateral temporal re-aggregation. We explore three schemes to build the aggregation: 1) a two-stage re-aggregation mechanism is applied to provide target prior to the current frame, which obtains more valid feature matching and information aggregation;2) a query-memory bilateral aggregation module is proposed to aggregate features from an unlimited amount of past frames and enable the mutual perception between different frames to validate the gathered information;3) we guide the learning of aggregation modules through a novel cross-task representation distillation, transferring the knowledge from a semi-supervised model to our weakly-supervised model without increasing the inference latency. These schemes collaboratively build an efficient and competent aggregation process, thus we can fully exploit the video context to make the inference. Experimental results on four benchmarks show that our method achieves superior performance than previous methods and still maintains the efficiency (e.g., overall scores of 70.4% and 72.5% on the YouTube-VOS and DAVIS 2017 validation sets, respectively).

关键词： video object segmentation motion segmentation temporal aggregation weakly-supervised prediction

来源：评论

学校读者我要写书评

暂无评论

Weakly supervised video object segmentation initialized with referring expression

引用

NEUROCOMPUTING 2021年 453卷 754-765页

作者： Bu, Xiaoqing Sun, Yukuan Wang, Jianming Liu, Kunliang Liang, Jiayu Jin, Guanghao Chung, Tae-Sun Tiangong Univ Sch Elect & Informat Engn Tianjin Peoples R China Ctr Engn Internship & Training Tianjin Peoples R China Tiangong Univ Tianjin Int Joint Res & Dev Ctr Autonomous Intell Tianjin Peoples R China Tiangong Univ Tianjin Key Lab Autonomous Intelligence Technol & Tianjin Peoples R China Tiangong Univ Sch Comp Sci & Technol Tianjin Peoples R China Ajou Univ Dept Software Suwon South Korea

With the aid of one manually annotated frame, One-Shot video object segmentation (OSVOS) uses a CNN architecture to tackle the problem of semi-supervised video object segmentation (VOS). However, annotating a pixel-level segmentation mask is expensive and time-consuming. To alleviate the problem, we explore a language interactive way of initializing semi-supervised VOS and run the semi-supervised methods into a weakly supervised mode. Our contributions are two folds: (i) we propose a variant of OSVOS initialized with referring expressions (REVOS), which locates a target object by maximizing the matching score between all the candidates and the referring expression;(ii) segmentation performance of semi-supervised VOS methods varies dramatically when selecting different frames for annotation. We present a strategy of the best annotation frame selection by using image similarity measurement. Meanwhile, we first to propose a multiple frame annotation selection strategy for initialization of semi-supervised VOS with more than one annotated frames. Finally we evaluate our method on DAVIS-2016 dataset, and experimental results show that REVOS achieves similar performance (79.94% measured by average IoU) compared with OSVOS (80.1%). Although current REVOS implementation is specific to the method of one-shot video object segmentation, it can be more widely applicable to other semi-supervised VOS methods. (c) 2020 Elsevier B.V. All rights reserved.

关键词： video object segmentation Referring Expression Natural Language Processing

来源：评论

学校读者我要写书评

暂无评论

Motion cues guided feature aggregation and enhancement for video object segmentation

引用

NEUROCOMPUTING 2022年 493卷 176-190页

作者： Li, Xuejun Zheng, Wenming Zong, Yuan Southeast Univ Sch Biol Sci & Med Engn Key Lab Child Dev & Learning Sci Minist Educ Nanjing Peoples R China

video object segmentation (VOS) aims to separate unknown target objects from various given video sequences. Although many recent successful methods boosted the performance of VOS, especially those using deep convolution neural networks (CNNs), it is still difficult to aggregate deep features as well as motion cues effectively, which can be important to associate valid information of adjacent frames in video sequences. To tackle this problem, we propose a simple yet effective feature optimization method for VOS based on motion information. To achieve this, we construct a two-branch deep network and use computed motion cues (i.e., optical flow) to jointly optimize global and local interframe correlation information. Additionally, a clustering-based feature enhancement module is proposed to further fuse motion information and enhance the feature saliency of the target area. Optimized feature maps show a significant performance improvement in the final VOS tasks, especially those with rapid target movement. Experiments on the DAVIS16, DAVIS17, YouTube-objects and YouTube-VOS datasets demonstrate that our simple feature aggregation and enhancement method for VOS improves segmentation accuracy effectively and gains an impressive result compared to many state-of-the-art methods. (c) 2022 Elsevier B.V. All rights reserved.

关键词： video object segmentation Convolutional neural networks Clustering Optical flow Feature fusion

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：