检索结果-内蒙古大学图书馆

2M3DF: Advancing 3D Industrial Defect Detection with Multi Perspective Multimodal Fusion Network

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Asad, Mujtaba Azeem, Waqar Jiang, He Mustafa, Hafiz Tayyab Yang, Jie Liu, Wei Shanghai Jiao Tong University Institute of Image Processing and Pattern Recognition Department of Automation Shanghai200240 China Lahore Garrison University Department of Software Engineering Lahore54000 Pakistan China University of Mining and Technology School of Information and Control Engineering Jiangsu Xuzhou221116 China Zhejiang Normal University School of Computer Science and Technology Jinhua321004 China

In the context of Industrial Anomaly Detection (IAD), ensuring the quality of manufactured products is critical. Traditional 2D based methods often fail to capture anomalies present in complex 3D shapes. For effective anomaly detection in 3D shapes, it is essential to incorporate global semantic context, local geometric structure, and color information of the object. To fully leverage these features, we propose a network named 2M3DF, that leverages knowledge from multi-view RGB images and corresponding point cloud information for enhanced anomaly detection performance. Our model initially employs pre-trained feature extractors that generate local features from multi-view RGB images and corresponding point clouds. The novel inter-modality feature representation and fusion module first adapts these inter-modality features and then effectively aligns and aggregates these multimodality features on a pixel-to-point basis. To learn the normality from point-wise fused multimodal features, we fit a multivariate Gaussian distribution to model the normal feature distribution. Comprehensive experimental evaluations using the MVTec3D-AD and Eyecandies dataset validate the effectiveness of our propose model and demonstrate significant improvements in comparison to existing state-of-the-art methods. Our model achieves a 96.6% mean I-AUROC while delivering real-time results. © 1991-2012 IEEE.

关键词： Normal distribution

来源：评论

学校读者我要写书评

暂无评论

Cross-Modal De-Deviation for Enhancing Few-Shot Classification

SSRN

引用

SSRN 2023年

作者： Pan, Mei-Hong Shen, Hong-Bin School of Electronic Information and Electrical Engineering Shanghai Jiaotong University China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai200240 China

Few-shot learning poses a critical challenge due to the deviation problem caused by the scarcity of available samples. In this work, we aim to address deviations in both feature representations and prototypes. To achieve this, we propose a cross-modal de-deviation framework that leverages class semantic information to provide robust prior knowledge for the samples. This framework begins with a visual-to-semantic autoencoder trained on the labeled samples to predict semantic features for the unlabeled samples. Then, we devise a binary linear programming model to incorporate the initial prototypes with the cluster centers of the unlabeled samples. To avoid mismatch between the cluster centers and the initial prototypes, we conduct the label assignment process in the semantic space using the class ground truth semantic features as the reference points, with the cluster centers transformed into semantic representations. Moreover, we model a linear classifier with the concatenation of the refined prototypes and the class ground truth semantic features serving as the initial weights. Then we propose a novel optimization strategy based on the alternating least squares (ALS) model. From the ALS model, we can derive two closed-form solutions regarding to the features and weights, facilitating alternative optimization of them. Extensive experiments conducted on three standard benchmarks demonstrate the competitive advantages of our CMDD method over the state-of-the-art few-shot classification methods, confirming its effectiveness in reducing deviation. © 2023, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Online Lidar-Camera Extrinsic Parameters Self-Checking

SSRN

引用

SSRN 2023年

作者： Wei, Pengjin Yan, Guohang You, Xin Fang, Kun Ma, Tao Yang, Jie Liu, Wei Department of Automation Shanghai Jiao Tong University 800 Dongchuan RD Minhang District China Autonomous Driving Group Shanghai AI Laboratory 129 Longwen RD Xuhui District China Institute of Image Processing and Pattern Recognition China

With the development of neural networks and the increasing popularity of automatic driving, the calibration of the LiDAR and the camera has attracted more and more attention. This calibration task is multi-modal, where the rich color and texture information captured by the camera and the accurate three-dimensional spatial information from the LiDAR is incredibly significant for downstream tasks. In real-world applications, as smart cars roll off the production line, their LiDAR and camera systems are meticulously calibrated, while in the rest of the car life period, the poses of the LiDARs and cameras no longer get continually supervised to ensure the security. To this end, this paper proposes a self-checking algorithm to judge whether the extrinsic parameters are well-calibrated during the car life period by introducing a binary classification network based on the fused information from the camera and the LiDAR. Moreover, since there is no such dataset for the task in this work, we further generate a new dataset branch from the KITTI dataset tailored for the task. Our experiments on the proposed dataset branch demonstrate the performance of our method. To the best of our knowledge, this is the first work to address the significance of continually checking the calibrated extrinsic parameters for autonomous driving. The dataset and code are available at https://***/OpenCalib/LiDAR2camera_self-check © 2023, The Authors. All rights reserved.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

MBD-Net: Multi-Branch Dilated Convolutional Network With Cyst Discriminator for Renal Multi-Structure Segmentation

MBD-Net: Multi-Branch Dilated Convolutional Network With Cys...

引用

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Yusheng Liu Yingjie Zhao Meihuan Wang Yichao Hao Xiuying Wang Lisheng Wang Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai China College of Medicine and Biological Information Engineering Northeastern University Shenyang China School of Computer Science The University of Sydney Sydney NSW Australia

In surgery-based renal cancer treatment, one of the most essential tasks is the three-dimensional (3D) kidney parsing on computed tomography angiography (CTA) images. In this paper, we propose an end-to-end convolutional neural network-based framework to segment multiple renal structures, including kidneys, kidney tumors, arteries, and veins from arterial-phase CT images. Our method consists of two collaborative modules: First, we propose an encoding-decoding network, named Multi-Branch Dilated Convolutional Network (MBD-Net), consisting of residual, hybrid dilated convolutional, and reduced-dimensional convolutional structures, which improves the feature extraction ability with relatively fewer network parameters. Given that renal tumors and cysts have confusing geometric structures, we also design the Cyst Discriminator to effectively distinguish tumors from cysts without labeling information via gray-scale curves and radiographic features. We have quantitatively evaluated our approach on a publicly available dataset from MICCAI 2022 Kidney Parsing for Renal Cancer Treatment Challenge (KiPA2022), with mean Dice similarity coefficient (DSC) as 96.18%, 90.99%, 88.66% and 80.35% for the kidneys, kidney tumors, arteries, and veins respectively, winning the stable and top performance in the *** relevance—The proposed CNN-Based framework can automatically segment 3D kidneys, renal tumors, arteries, and veins for kidney parsing techniques, benefiting surgery-based renal cancer treatment.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Face Forgery Detection Algorithm Based on Improved MobileViT Network

Face Forgery Detection Algorithm Based on Improved MobileViT...

引用

6th International Conference on Intelligent Computing and Signal Processing (ICSP)

作者： Tiantian Wang Xiaoqi Lu School of Information Engineering Inner Mongolia University of Science and Technology Baotou China Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Processing Baotou China Institute of Information Engineering Inner Mongolia University of Technology Hohhot China

DeepFakes blur the boundaries between reality and forgery, resulting in the collapse of exiting credit system, causing immeasurable consequences for national security and social order. Through analysis of existing face forgery techniques, it is found that most generation techniques rely on random noise distribution, and global information will be lost after up sampling. Therefore, we propose a deepfake detection algorithm based on improved MobileViT, which uses CNN local space biasing and the global space representation of the Transformer network to learn the local features and global representation of forged faces, respectively. Coordinate attention is introduced to obtain directional perception and position sensitive information, making the model locate synthetic traces of fake faces better and fusion local and global representation more effectively. For the improved generalization of the model, with the GELU activation function to solve the problem of neuron death. Our model achieved 96.2% on FF++(C23) datasets, and 93.7%,94.1%,96.3%,87.9% on DF, F2F, FS, and NT datasets, respectively. Comparing with previous methods, our model has shown detection robustness and better generalization.

关键词：

来源：评论

学校读者我要写书评

暂无评论

FastFlowNet: A lightweight network for fast optical flow estimation

arXiv

引用

arXiv 2021年

作者： Kong, Lingtong Shen, Chunhua Yang, Jie Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China The University of Adelaide Australia

Dense optical flow estimation plays a key role in many robotic vision tasks. It has been predicted with satisfying accuracy than traditional methods with advent of deep learning. However, current networks often occupy large number of parameters and require heavy computation costs. These drawbacks have hindered applications on power- or memory-constrained mobile devices. To deal with these challenges, in this paper, we dive into designing efficient structure for fast and accurate optical flow prediction. Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations. First, a new head enhanced pooling pyramid (HEPP) feature extractor is employed to intensify high-resolution pyramid feature while reducing parameters. Second, we introduce a novel center dense dilated correlation (CDDC) layer for constructing compact cost volume that can keep large search radius with reduced computation burden. Third, an efficient shuffle block decoder (SBD) is implanted into each pyramid level to acclerate flow estimation with marginal drops in accuracy. Experiments on both synthetic Sintel and real-world KITTI datasets demonstrate the effectiveness of proposed approaches, which consumes only 1/10 computation of comparable networks to get 90% of their performance. In particular, FastFlowNet only contains 1.37 M parameters and runs at 90 or 5.7 fps with one desktop NVIDIA GTX 1080 Ti or embedded Jetson TX2 GPU on Sintel resolution images. Copyright © 2021, The Authors. All rights reserved.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Noise Perturbation Based Graph Contrastive Learning via Flexible Filters for Node Classification

Noise Perturbation Based Graph Contrastive Learning via Flex...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Zhilong Xiong Jia Cai Ranhui Yan Xiaolin Huang xFusion Digital Technologies Company Limited Shenzhen China School of Digital Economics Guangdong University of Finance & Economics Guangzhou China School of Statistics and Mathematics Guangdong University of Finance & Economics Guangzhou China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Graph neural networks (GNNs), as a powerful deep learning framework for modeling graph-structured data, have attracted lots of attention recently. Most of existing GNNs need a lot of labeled data. However, constructing generalizable and robust representation from unlabeled graph data remains a challenge for GNNs. Existing graph contrastive learning (GCL) methods either try to uniformly drop edges, or intend to remove unimportant nodes and edges, which heavily relies on the specific structure of the data. Another thing is that vanilla graph convolutional network only utilize low-pass filter (adjacency matrix), which ignores the middle and high frequency information of the graph structural data. To tackle existing challenges in the GCL methods, instead, we propose a noise perturbation based general GCL framework via flexible filters. Specifically, we first add various types of noise to the nodes and edges. Subsequently, we design flexible filters, which are the combination of low, middle and high-pass filters. Our investigation systematically examines the impact of noise and filters, with an initial theoretical analysis linking these elements to the triplet loss function, shedding light on their roles. Extensive experiments in node classification showcase that our proposed approach surpasses existing state-of-the-art baselines. Surprisingly, we find that moderate levels of noise effectively alleviate the over-smoothing problem encountered in GNNs, while the use of flexible filters notably enhances model performance.

关键词： Filters Sensitivity analysis Perturbation methods image processing image edge detection Noise Low-pass filters

来源：评论

学校读者我要写书评

暂无评论

Optimal Initialization Conditions Discovery to Improve Clustering Based image Segmentation

SSRN

引用

SSRN 2022年

作者： Khan, Zubair Khan, Tehreem Yang, Jie Tu, Enmei Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai China Institute of Medical Robotics Shanghai Jiao Tong University Shanghai China University of Central Punjab Bahawalpur Pakistan

A B S T R A C TIn this paper, we propose a solution to the most important factor that deteriorates the performance of the clustering algorithms. We propose an efficient approach for discovering optimal initialization conditions for clustering-driven high-quality image segmentation (EAIS). The proposed approach solves the key clustering issue by uniquely utilizing image histograms to determine the optimal initialization conditions for pixel clusters (segmentation). The proposed approach comprises five modules, deep image reconstruction, intra-histogram peaks determination, inter-histogram peaks association, mutual consensus-oriented cluster seeds merging, and morphological reconstruction-driven spatial post-processing. Diverse experimental results on the BSDS500 benchmark validate that our proposed approach outperforms state-of-the-art (SOTA) methods regarding segmentation quality and computational efficiency. © 2022, The Authors. All rights reserved.

关键词： Merging

来源：评论

学校读者我要写书评

暂无评论

Isoform Function Prediction Based on Heterogeneous Graph Attention Networks

Isoform Function Prediction Based on Heterogeneous Graph Att...

引用

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Guo, Kuo Li, Yifan Chen, Hao Shen, Hong-Bin Yang, Yang Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Department of Computer Science and Engineering Shanghai200240 China Shanghai Jiao Tong University Key Laboratory of System Control and Information Processing Ministry of Education of China Institute of Image Processing and Pattern Recognition Shanghai200240 China Carnegie Mellon University School of Computer Science Computational Biology Department PittsburghPA15213 United States

ISBN: (纸本)9798350337488

Isoforms refer to different mRNA molecules transcribed from the same gene, which can be translated into proteins with varying structures and functions. Predicting the functions of isoforms is an essential topic in bioinformatics as it can provide valuable insights into the intricate mechanisms of gene regulation and biological processes. Conventionally, gene function labels are standardized in Gene Ontology (GO) terms. However, traditional methods for predicting isoform function are largely limited by the absence of isoform-specific labels, sparse annotations, and the vast number of GO terms. To address these issues, we propose HANIso, a deep learning-based method for isoform function prediction. HANIso leverages a pretrained protein language model to extract features from protein sequences. It also integrates heterogeneous information, such as isoform sequence features, GO annotations, and isoform interaction data, using a Heterogeneous Graph Attention Network (HAN). This allows the model to learn the importance of different sources of information and their semantic relationships through the attention mechanism. Our method can predict function labels at both the gene level and isoform level. We conduct experiments on two species datasets, and the results demonstrate that our method outperforms existing methods on both AUROC and AUPRC. HANIso has the potential to overcome the limitations of traditional methods and provide a more accurate and comprehensive understanding of isoform function. © 2023 IEEE.

关键词： alternative splicing gene ontology heterogeneous graph attention network isoform function prediction protein language model

来源：评论

学校读者我要写书评

暂无评论

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

FastFlowNet: A Lightweight Network for Fast Optical Flow Est...

引用

IEEE International Conference on Robotics and Automation (ICRA)

作者： Lingtong Kong Chunhua Shen Jie Yang Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China The University of Adelaide Australia

Dense optical flow estimation plays a key role in many robotic vision tasks. In the past few years, with the advent of deep learning, we have witnessed great progress in optical flow estimation. However, current networks often consist of a large number of parameters and require heavy computation costs, largely hindering its application on low power-consumption devices such as mobile phones. In this paper, we tackle this challenge and design a lightweight model for fast and accurate optical flow prediction. Our proposed FastFlowNet follows the widely-used coarse-to-fine paradigm with following innovations. First, a new head enhanced pooling pyramid (HEPP) feature extractor is employed to intensify high-resolution pyramid features while reducing parameters. Second, we introduce a new center dense dilated correlation (CDDC) layer for constructing compact cost volume that can keep large search radius with reduced computation burden. Third, an efficient shuffle block decoder (SBD) is implanted into each pyramid level to accelerate flow estimation with marginal drops in accuracy. Experiments on both synthetic Sintel data and real-world KITTI datasets demonstrate the effectiveness of the proposed approach, which needs only ¹/10 computation of comparable networks to achieve on par accuracy. In particular, FastFlowNet only contains 1.37M parameters; and can execute at 90 FPS (with a single GTX 1080Ti) or 5.7 FPS (embedded Jetson TX2 GPU) on a pair of Sintel images of resolution 1024 × *** is available at: https://***/fastflow

关键词： Technological innovation Costs Head Correlation Estimation Predictive models Feature extraction

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：