检索结果-内蒙古大学图书馆

Joint Semantic and Instance Segmentation in 3D Point Cloud Based on Transformer

学校读者我要写书评

暂无评论

Joint Semantic and Instance Segmentation in 3D Point Cloud B...

第35届中国控制与决策会议

作者： Suyi Liu Chengdong Wu Fang Xu Junxiang Wang Jianning Chi Xiaosheng Yu Faculty of Robot Science and Engineering Northeastern University Academia Sinica Shenyang Siasun Robot Automation Co. Ltd

Semantic and instance segmentation in 3 D point cloud have made tremendous progress in recent *** most current methods,semantic and instance segmentation are studied separately,but fall to integrate *** addition,the irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud *** this paper,we propose a joint segmentation method based on Transformer that able to make the two tasks take advantage of each other,leading to a win-win *** novel Transformer architecture is used to capture long-short contexts and demonstrates higher segmentation ***,for each query point,we use two different key sampling strategies:nearby dense points sampling and distant sparse points ***,matrix multiplication with query results in two different attention maps,which enables the model to enlarge the effective receptive ***,to make the two tasks take advantage of each other,we first combine the semantic perception abstracted from the high-dimensional semantic features with the instance features to enhance instance segmentation ***,the k-nearest neighbor(KNN) clustering method is used in the instance embedding to find a fixed number of adjacent points for each point in the instance clustering *** points of the same class are close to each other,the points of different classes are far away from each other,so as to realize the semantic segmentation model of instance *** experiments demonstrate the effectiveness and superiority of our method on S3 DIS and ShapeNet datasets.

关键词：

Fully Asynchronous Neuromorphic Perception for Mobile robot Dodging with Loihi Chips

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jiang, Junjie Kong, Delei Hu, Chenming Fang, Zheng Faculty of Robot Science and Engineering Northeastern University Shenyang China School of Robotics Hunan University Changsha China

Sparse and asynchronous sensing and processing in natural organisms lead to ultra low-latency and energy-efficient perception. Event cameras, known as neuromorphic vision sensors, are designed to mimic these characteristics. However, fully utilizing the sparse and asynchronous event stream remains challenging. Influenced by the mature algorithms of standard cameras, most existing event-based algorithms still rely on the"group of events" processing paradigm (e.g., event frames, 3D voxels) when handling event streams. This paradigm encounters issues such as feature loss, event stacking, and high computational burden, which deviates from the intended purpose of event cameras. To address these issues, we propose a fully asynchronous neuromorphic paradigm that integrates event cameras, spiking networks, and neuromorphic processors (Intel Loihi). This paradigm can faithfully process each event asynchronously as it arrives, mimicking the spike-driven signal processing in biological brains. We compare the proposed paradigm with the existing"group of events" processing paradigm in detail on the real mobile robot dodging task. Experimental results show that our scheme exhibits better robustness than frame-based methods with different time windows and light conditions. Additionally, the energy consumption per inference of our scheme on the embedded Loihi processor is only 4.30% of that of the event spike tensor method on NVIDIA Jetson Orin NX with energy-saving mode, and 1.64% of that of the event frame method on the same neuromorphic processor. As far as we know, this is the first time that a fully asynchronous neuromorphic paradigm has been implemented for solving sequential tasks on real mobile robot. © 2024, CC BY.

关键词： Mobile robots

EV-MGDispNet: Motion-Guided Event-Based Stereo Disparity Estimation Network with Left-Right Consistency

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jiang, Junjie Zhuang, Hao Huang, Xinjie Kong, Delei Fang, Zheng Faculty of Robot Science and Engineering Northeastern University Shenyang China School of Robotics Hunan University Changsha China

Event cameras have the potential to revolutionize the field of robot vision, particularly in areas like stereo disparity estimation, owing to their high temporal resolution and high dynamic range. Many studies use deep learning for event camera stereo disparity estimation. However, these methods fail to fully exploit the temporal information in the event stream to acquire clear event representations. Additionally, there is room for further reduction in pixel shifts in the feature maps before constructing the cost volume. In this paper, we propose EV-MGDispNet, a novel event-based stereo disparity estimation method. Firstly, we propose an edge-aware aggregation (EAA) module, which fuses event frames and motion confidence maps to generate a novel clear event representation. Then, we propose a motion-guided attention (MGA) module, where motion confidence maps utilize deformable transformer encoders to enhance the feature map with more accurate edges. Finally, we also add a census left-right consistency loss function to enhance the left-right consistency of stereo event representation. Through conducting experiments within challenging real-world driving scenarios, we validate that our method outperforms currently known state-of-the-art methods in terms of mean absolute error (MAE) and root mean square error (RMSE) metrics. Copyright © 2024, The Authors. All rights reserved.

关键词： Mean square error

Group-Mix SAM: Lightweight Segmentation Solution for Industrial Implementations

学校读者我要写书评

暂无评论

Group-Mix SAM: Lightweight Segmentation Solution for Industr...

Cognitive Computing and Complex Data (ICCD), International Conference on the

作者： Liang Wu Puhong Duan Xiaoguang Ma Faculty of Robot Science and Engineering Northeastern University Shenyang China School of Robotics Hunan University Changsha China

ISBN: (数字)9798350352894

ISBN: (纸本)9798350352900

Since its emergence in early 2023, Segment Anything Model(SAM) has garnered significant academic interests and spawned many investigations from various perspectives due to its zero-shot generalization ability. However, the deployment of SAM in scenes that required real-time performance, such as industrial assembly liners, had severe challenges due to its heavy image encoder, whose parameters were as large as 632M. In this study, we proposed a lightweight segmentation solution for industrial assembly line applications, wherein the heavyweight image encoder was replaced with a lightweight one. Specifically, we employed decoupled distillation to train the encoder of MobileSAM in a resource-limited setting. The entire knowledge distillation experiment could be completed in a single day on a single RTX 4090. The resulting lightweight SAM, called Group- Mix SAM, had 37.63% (2.16M) fewer parameters and 42.5% (15614.7M) fewer floating-point operations compared to MobileSAM. Additionally, on our homemade industrial dataset: MALD, its mIoU was only marginally lower than that of MobileSAM, at 0.615. Finally, we conducted comprehensive experiments to demonstrate the superiority of Group-Mix SAM in real industrial scenes.

关键词： Image segmentation Computational modeling Computer architecture Real-time systems Data models Cognitive systems Assembly

TSCM: A Teacher-Student Model for Visual Place Recognition Using Cross-Metric Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Shen, Yehui Liu, Mingmin Lu, Huimin Chen, Xieyuanli Faculty of Robot Science and Engineering Northeastern University China College of Intelligence Science and Technology National University of Defense Technology China SIASUN Robot& Automation Co. Ltd China

Visual place recognition (VPR) plays a pivotal role in autonomous exploration and navigation of mobile robots within complex outdoor environments. While cost-effective and easily deployed, camera sensors are sensitive to lighting and weather changes, and even slight image alterations can greatly affect VPR efficiency and precision. Existing methods overcome this by exploiting powerful yet large networks, leading to significant consumption of computational resources. In this paper, we propose a high-performance teacher and lightweight student distillation framework called TSCM. It exploits our devised cross-metric knowledge distillation to narrow the performance gap between the teacher and student models, maintaining superior performance while enabling minimal computational load during deployment. We conduct comprehensive evaluations on large-scale datasets, namely Pittsburgh30k and Pittsburgh250k. Experimental results demonstrate the superiority of our method over baseline models in terms of recognition accuracy and model parameter efficiency. Moreover, our ablation studies show that the proposed knowledge distillation technique surpasses other counterparts. The code of our method has been released at https://***/nubot-nudt/TSCM. Copyright © 2024, The Authors. All rights reserved.

关键词： Students

EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Jiaxu He, Junhao Zhang, Ziyi Sun, Mingyuan Sun, Jingkai Xu, Renjing Function Hub Hong Kong University of Science and Technology Guangzhou China Faculty of Robot Science and Engineering Northeastern University Shenyang China

Event cameras offer promising advantages such as high dynamic range and low latency, making them well-suited for challenging lighting conditions and fast-moving scenarios. However, reconstructing 3D scenes from raw event streams is difficult because event data is sparse and does not carry absolute color information. To release its potential in 3D reconstruction, we propose the first event-based generalizable 3D reconstruction framework, called EvGGS, which reconstructs scenes as 3D Gaussians from only event input in a feedforward manner and can generalize to unseen cases without any retraining. This framework includes a depth estimation module, an intensity reconstruction module, and a Gaussian regression module. These submodules connect in a cascading manner, and we collaboratively train them with a designed joint loss to make them mutually promote. To facilitate related studies, we build a novel event-based 3D dataset with various material objects and calibrated labels of grayscale images, depth maps, camera poses, and silhouettes. Experiments show models that have jointly trained significantly outperform those trained individually. Our approach performs better than all baselines in reconstruction quality, and depth/intensity predictions with satisfactory rendering speed. Code and Dataset are demonstrated https://***/Mercerai/EvGGS/ Copyright © 2024, The Authors. All rights reserved.

关键词： Gaussian distribution

TSCM: A Teacher-Student Model for Vision Place Recognition Using Cross-Metric Knowledge Distillation

学校读者我要写书评

暂无评论

TSCM: A Teacher-Student Model for Vision Place Recognition U...

IEEE International Conference on robotics and Automation (ICRA)

作者： Yehui Shen Mingmin Liu Huimin Lu Xieyuanli Chen Faculty of Robot Science and Engineering Northeastern University China SIASUN Robot& Automation Co. Ltd China College of Intelligence Science and Technology National University of Defense Technology China

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

关键词： Visualization Accuracy Navigation Computational modeling robot vision systems Real-time systems Sensors

A Hand Action Recognition Framework Based on Multimodal Foundation Model and Tracking of Skeleton Point Trajectorie

学校读者我要写书评

暂无评论

A Hand Action Recognition Framework Based on Multimodal Foun...

Cognitive Computing and Complex Data (ICCD), International Conference on the

作者： Liang Wu Xiaoyu Jiang lXiaoguang Ma Faculty of Robot Science and Engineering Northeastern University Shenyang China Department of Systems Engineering City University of Hong Kong Hong Kong

ISBN: (数字)9798350352894

ISBN: (纸本)9798350352900

Although accurate hand motion recognition was particularly crucial in monitoring modern industrial assembly lines, it still faced challenges, such as high-cost training datasets and poor real-time performance. In this paper, we propose an innovative paradigm for recognizing hand actions on industrial assembly lines, wherein the problem of hand action recognition was transformed into a hand skeletal point trajectory classification problem. This solved the problem of the requirement for a large dataset and poor real-time performance for action recognition on the assembly line. Comprehensive experiments were concluded to demonstrate the effectiveness and superiority of the method, wherein the accuracy of hand action recognition reached 98.8% with the proposed methods being implemented in real industrial assembly lines of Midea Group.

关键词： Hands Training Accuracy Tracking Foundation models Real-time systems Skeleton Trajectory Assembly Monitoring

Temporal Enhance and Spatial Gated Network for Group Activity Recognition

学校读者我要写书评

暂无评论

Temporal Enhance and Spatial Gated Network for Group Activit...

第34届中国控制与决策会议

作者： Tiansheng Sun Jianning Chi Chengdong Wu Faculty of Robot Science and Engineering Northeastern University

The main goal of Group Activity Recognition(GAR) is to analyze the group behavior in a multi-actors *** is a challenging task that integrates group activity features from individual actors who frequently interact with each ***,existing state-of-the-art methods do not consider temporal information and take full advantage of spatial ***,existing methods utilize max-pooling operation to generate group activity features,which introduces noise into group activity features and leads to bad *** tackle these problems,a temporal enhance and spatial gated network is *** fully utilize the rich temporal features and spatial features,two branches are set up to extract temporal features and actor interactive relations,***,a feature aggregation module is used to generate group activity features,which can decrease the noise introduced by the max-pooling *** conduct a series of experiments on two stand *** results on these benchmarks demonstrate that the proposed approach achieves state-of-the-art *** with previous work,the recognition accuracy increases by 0.5%～1.0%.

关键词： Group Activity Recognition Convolutional Neural Network Interactive Relations