检索结果-内蒙古大学图书馆

FastFlowNet: A lightweight network for fast optical flow estimation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Kong, Lingtong Shen, Chunhua Yang, Jie Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China The University of Adelaide Australia

Dense optical flow estimation plays a key role in many robotic vision tasks. It has been predicted with satisfying accuracy than traditional methods with advent of deep learning. However, current networks often occupy large number of parameters and require heavy computation costs. These drawbacks have hindered applications on power- or memory-constrained mobile devices. To deal with these challenges, in this paper, we dive into designing efficient structure for fast and accurate optical flow prediction. Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations. First, a new head enhanced pooling pyramid (HEPP) feature extractor is employed to intensify high-resolution pyramid feature while reducing parameters. Second, we introduce a novel center dense dilated correlation (CDDC) layer for constructing compact cost volume that can keep large search radius with reduced computation burden. Third, an efficient shuffle block decoder (SBD) is implanted into each pyramid level to acclerate flow estimation with marginal drops in accuracy. Experiments on both synthetic Sintel and real-world KITTI datasets demonstrate the effectiveness of proposed approaches, which consumes only 1/10 computation of comparable networks to get 90% of their performance. In particular, FastFlowNet only contains 1.37 M parameters and runs at 90 or 5.7 fps with one desktop NVIDIA GTX 1080 Ti or embedded Jetson TX2 GPU on Sintel resolution images. Copyright © 2021, The Authors. All rights reserved.

关键词： Deep learning

Online Lidar-Camera Extrinsic Parameters Self-Checking

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Wei, Pengjin Yan, Guohang You, Xin Fang, Kun Ma, Tao Yang, Jie Liu, Wei Department of Automation Shanghai Jiao Tong University 800 Dongchuan RD Minhang District China Autonomous Driving Group Shanghai AI Laboratory 129 Longwen RD Xuhui District China Institute of Image Processing and Pattern Recognition China

With the development of neural networks and the increasing popularity of automatic driving, the calibration of the LiDAR and the camera has attracted more and more attention. This calibration task is multi-modal, where the rich color and texture information captured by the camera and the accurate three-dimensional spatial information from the LiDAR is incredibly significant for downstream tasks. In real-world applications, as smart cars roll off the production line, their LiDAR and camera systems are meticulously calibrated, while in the rest of the car life period, the poses of the LiDARs and cameras no longer get continually supervised to ensure the security. To this end, this paper proposes a self-checking algorithm to judge whether the extrinsic parameters are well-calibrated during the car life period by introducing a binary classification network based on the fused information from the camera and the LiDAR. Moreover, since there is no such dataset for the task in this work, we further generate a new dataset branch from the KITTI dataset tailored for the task. Our experiments on the proposed dataset branch demonstrate the performance of our method. To the best of our knowledge, this is the first work to address the significance of continually checking the calibrated extrinsic parameters for autonomous driving. The dataset and code are available at https://***/OpenCalib/LiDAR2camera_self-check © 2023, The Authors. All rights reserved.

关键词： Cameras

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

学校读者我要写书评

暂无评论

FastFlowNet: A Lightweight Network for Fast Optical Flow Est...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Lingtong Kong Chunhua Shen Jie Yang Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China The University of Adelaide Australia

Dense optical flow estimation plays a key role in many robotic vision tasks. In the past few years, with the advent of deep learning, we have witnessed great progress in optical flow estimation. However, current networks often consist of a large number of parameters and require heavy computation costs, largely hindering its application on low power-consumption devices such as mobile phones. In this paper, we tackle this challenge and design a lightweight model for fast and accurate optical flow prediction. Our proposed FastFlowNet follows the widely-used coarse-to-fine paradigm with following innovations. First, a new head enhanced pooling pyramid (HEPP) feature extractor is employed to intensify high-resolution pyramid features while reducing parameters. Second, we introduce a new center dense dilated correlation (CDDC) layer for constructing compact cost volume that can keep large search radius with reduced computation burden. Third, an efficient shuffle block decoder (SBD) is implanted into each pyramid level to accelerate flow estimation with marginal drops in accuracy. Experiments on both synthetic Sintel data and real-world KITTI datasets demonstrate the effectiveness of the proposed approach, which needs only ¹/10 computation of comparable networks to achieve on par accuracy. In particular, FastFlowNet only contains 1.37M parameters; and can execute at 90 FPS (with a single GTX 1080Ti) or 5.7 FPS (embedded Jetson TX2 GPU) on a pair of Sintel images of resolution 1024 × *** is available at: https://***/fastflow

关键词： Technological innovation Costs Head Correlation Estimation Predictive models Feature extraction

CDFI: Cross Domain Feature Interaction for Robust Bronchi Lumen Detection

学校读者我要写书评

暂无评论

CDFI: Cross Domain Feature Interaction for Robust Bronchi Lu...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Jiasheng Xu Tianyi Zhang Yangqian Wu Jie Yang Guang–Zhong Yang Yun Gu Institute of Medical Robotics Shanghai Jiao Tong University Shanghai CHINA Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai CHINA MOE Key Laboratory of System Control and Information Processing Shanghai P.R. China Shanghai Center for Brain Science and Brain-Inspired Technology Shanghai China

Endobronchial intervention is increasingly used as a minimally invasive means for the treatment of pulmonary diseases. In order to reduce the difficulty of manipulation in complex airway networks, robust lumen detection is essential for intraoperative guidance. However, these methods are sensitive to visual artifacts which are inevitable during the surgery. In this work, a cross domain feature interaction (CDFI) network is proposed to extract the structural features of lumens, as well as to provide artifact cues to characterize the visual features. To effectively extract the structural and artifact features, the Quadruple Feature Constraints (QFC) module is designed to constrain the intrinsic connections of samples with various imaging-quality. Furthermore, we design a Guided Feature Fusion (GFF) module to supervise the model for adaptive feature fusion based on different types of artifacts. Results show that the features extracted by the proposed method can preserve the structural information of lumen in the presence of large visual variations, bringing much-improved lumen detection accuracy.

关键词：

SRSNetwork: Siamese Reconstruction-Segmentation Networks based on Dynamic-Parameter Convolution

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Nian, Bingkun Tang, Fenghe Ding, Jianrui Zhang, Pingping Yang, Jie Kevin Zhou, S. Liu, Wei Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China School of Biomedical Engineering Suzhou Institute for Advanced Research University of Science and Technology of China China School of Computer Science and Technology Harbin Institute of Technology China School of artificial intelligence Dalian University of Technology China

In this paper, we present a high-performance deep neural network for weak target image segmentation, including medical image segmentation and infrared image segmentation. To this end, this work analyzes the existing dynamic convolutions and proposes dynamic parameter convolution (DPConv). Furthermore, it reevaluates the relationship between reconstruction tasks and segmentation tasks from the perspective of DPConv, leading to the proposal of a dual-network model called the Siamese Reconstruction-Segmentation Network (SRSNet). The proposed model is not only a universal network but also enhances the segmentation performance without altering its structure, leveraging the reconstruction task. Additionally, as the amount of training data for the reconstruction network increases, the performance of the segmentation network also improves synchronously. On seven datasets including five medical datasets and two infrared image datasets, our SRSNet consistently achieves the best segmentation results. The code is released at https://***/fidshu/SRSNet. Copyright © 2023, The Authors. All rights reserved.

关键词： Convolution

Modeling Inter-Intra Heterogeneity for Graph Federated Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Wentao Chen, Shuo Tong, Yongxin Gu, Tianlong Gong, Chen School of Computer Science and Engineering Nanjing University of Science and Technology China Center for Advanced Intelligence Project RIKEN Japan State Key Laboratory of Complex & Critical Software Environment Beihang University China Jinan University China Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

Heterogeneity is a fundamental and challenging issue in federated learning, especially for the graph data due to the complex relationships among the graph nodes. To deal with the heterogeneity, lots of existing methods perform the weighted federation based on their calculated similarities between pairwise clients (i.e., subgraphs). However, their inter-subgraph similarities estimated with the outputs of local models are less reliable, because the final outputs of local models may not comprehensively represent the real distribution of subgraph data. In addition, they ignore the critical intra-heterogeneity which usually exists within each subgraph itself. To address these issues, we propose a novel Federated learning method by integrally modeling the Inter-Intra Heterogeneity (FedIIH). For the inter-subgraph relationship, we propose a novel hierarchical variational model to infer the whole distribution of subgraph data in a multi-level form, so that we can accurately characterize the inter-subgraph similarities with the global perspective. For the intra-heterogeneity, we disentangle the subgraph into multiple latent factors and partition the model parameters into multiple parts, where each part corresponds to a single latent factor. Our FedIIH not only properly computes the distribution similarities between subgraphs, but also learns disentangled representations that are robust to irrelevant factors within subgraphs, so that it successfully considers the inter- and intra- heterogeneity simultaneously. Extensive experiments on six homophilic and five heterophilic graph datasets in both non-overlapping and overlapping settings demonstrate the effectiveness of our method when compared with nine state-of-the-art methods. Specifically, FedIIH averagely outperforms the second-best method by a large margin of 5.79% on all heterophilic datasets. Code is available at https://***/blgpb/FedIIH. Copyright © 2024, The Authors. All rights reserved.

关键词： Federated learning

Robot Debater: Debate-styled Text Auto-generation System Based on Large Foundation Language Models

学校读者我要写书评

暂无评论

Robot Debater: Debate-styled Text Auto-generation System Bas...

pattern recognition and Machine Learning (PRML), IEEE International Conference on

作者： Yu Zhu Yijun Ling Xufeng Ling Jie Yang Shanghai Library (Institute of Scientific and Technical Information of Shanghai) Shanghai China Shanghai Xiangming High School Shanghai China School of Artificial Intelligence Shanghai Normal University Tianhua College Shanghai China Institute of Image Processing and Pattern Recognition Shanghai Jiaotong University Shanghai China

We use a large foundation language model, which is fine-tuned with debate corpora, to develop a robot debater application. To address the limitations of requiring immense computational power in large base language models, this study takes advantage of the Low Rank Adaption characteristic prevalent in domain expert knowledge. By applying Low Rank Adaption technology and fine-tuning with a dedicated dataset, the computational load is reduced to just one-thousandth of what is needed for a large language model, greatly expanding the application scenarios of robot debaters using large language models. In view of the characteristics of debate competitions, this model can preset a variety of debate scenarios and supports personalized debate processes. It employs intelligent voice recognition technology combined with a multi-channel voice input method, allowing for precise localization of different human debaters and improving the accuracy of voice input recognition. The system can support multiple large-scale language generation models and utilize various different voice broadcasting systems, including male and female voice styles, as well as a range of voice emotions. This model can be applied to debate competitions held in universities, high schools, and various industries. It can support human-machine debates as well as machine-to-machine debates.

关键词：

Hybrid Data-Free Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tang, Jialiang Chen, Shuo Gong, Chen School of Computer Science and Engineering Nanjing University of Science and Technology China Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education China Jiangsu Key Laboratory of Image and Video Understanding for Social Security China Center for Advanced Intelligence Project RIKEN Japan Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

Data-free knowledge distillation aims to learn a compact student network from a pre-trained large teacher network without using the original training data of the teacher network. Existing collection-based and generation-based methods train student networks by collecting massive real examples and generating synthetic examples, respectively. However, they inevitably become weak in practical scenarios due to the difficulties in gathering or emulating sufficient real-world data. To solve this problem, we propose a novel method called Hybrid Data-Free Distillation (HiDFD), which leverages only a small amount of collected data as well as generates sufficient examples for training student networks. Our HiDFD comprises two primary modules, i.e., the teacher-guided generation and student distillation. The teacher-guided generation module guides a Generative Adversarial Network (GAN) by the teacher network to produce high-quality synthetic examples from very few real-world collected examples. Specifically, we design a feature integration mechanism to prevent the GAN from overfitting and facilitate the reliable representation learning from the teacher network. Meanwhile, we drive a category frequency smoothing technique via the teacher network to balance the generative training of each category. In the student distillation module, we explore a data inflation strategy to properly utilize a blend of real and synthetic data to train the student network via a classifier-sharing-based feature alignment technique. Intensive experiments across multiple benchmarks demonstrate that our HiDFD can achieve state-of-the-art performance using 120 times less collected data than existing methods. Code is available at https://***/tangjialiang97/HiDFD. Copyright © 2024, The Authors. All rights reserved.

关键词： Students

Application of an Improved Focal Loss in Vehicle Detection 19th

学校读者我要写书评

暂无评论

Application of an Improved Focal Loss in Vehicle Detection

19th International Conference on Artificial Intelligence and Soft Computing, ICAISC 2020

作者： He, Xuanlin Yang, Jie Kasabov, Nikola Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai China Auckland University of Technology Auckland New Zealand

ISBN: (纸本)9783030614003

Object detection is an important and fundamental task in computer vision. Recently, the emergence of deep neural network has made considerable progress in object detection. Deep neural network object detectors can be grouped in two broad categories: the two-stage detector and the one-stage detector. One-stage detectors are faster than two-stage detectors. However, they suffer from a severe foreground-backg-round class imbalance during training that causes a low accuracy performance. RetinaNet is a one-stage detector with a novel loss function named Focal Loss which can reduce the class imbalance effect. Thereby RetinaNet outperforms all the two-stage and one-stage detectors in term of accuracy. The main idea of focal loss is to add a modulating factor to rectify the cross-entropy loss, which down-weights the loss of easy examples during training and thus focuses on the hard examples. However, cross-entropy loss only focuses on the loss of the ground-truth classes and thus it can’t gain the loss feedback from the false classes. Thereby cross-entropy loss does not achieve the best convergence. In this paper, we proposed a new loss function named Dual Cross-Entropy Focal Loss, which improves on the focal loss. Dual cross-entropy focal loss adds a modulating factor to rectify the dual cross-entropy loss towards focusing on the hard samples. Dual cross-entropy loss is an improved variant of cross-entropy loss, which gains the loss feedback from both the ground-truth classes and the false classes. We changed the loss function of RetinaNet from focal loss to our dual cross-entropy focal loss and performed some experiments on a small vehicle dataset. The experimental results show that our new loss function improves the vehicle detection performance. © 2020, Springer Nature Switzerland AG.

关键词： Object detection