检索结果-内蒙古大学图书馆

Learning Parallax Transformer Network for Stereo Image JPEG Artifacts Removal

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jiang, Xuhao Tan, Weimin Cheng, Ri Zhou, Shili Yan, Bo School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai China

Under stereo settings, the performance of image JPEG artifacts removal can be further improved by exploiting the additional information provided by a second view. However, incorporating this information for stereo image JPEG artifacts removal is a huge challenge, since the existing compression artifacts make pixel-level view alignment difficult. In this paper, we propose a novel parallax transformer network (PTNet) to integrate the information from stereo image pairs for stereo image JPEG artifacts removal. Specifically, a well-designed symmetric bi-directional parallax transformer module is proposed to match features with similar textures between different views instead of pixel-level view alignment. Due to the issues of occlusions and boundaries, a confidence-based cross-view fusion module is proposed to achieve better feature fusion for both views, where the cross-view features are weighted with confidence maps. Especially, we adopt a coarse-to-fine design for the cross-view interaction, leading to better performance. Comprehensive experimental results demonstrate that our PTNet can effectively remove compression artifacts and achieves superior performance than other testing state-of-the-art methods. © 2022, CC BY.

关键词： Textures

Geometry-Aware Reference Synthesis for Multi-View Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cheng, Ri Sun, Yuqi Yan, Bo Tan, Weimin Ma, Chenxi School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai China

Recent multi-view multimedia applications struggle between high-resolution (HR) visual experience and storage or bandwidth constraints. Therefore, this paper proposes a Multi-View Image Super-Resolution (MVISR) task. It aims to increase the resolution of multiview images captured from the same scene. One solution is to apply image or video super-resolution (SR) methods to reconstruct HR results from the low-resolution (LR) input view. However, these methods cannot handle large-angle transformations between views and leverage information in all multi-view images. To address these problems, we propose the MVSRnet, which uses geometry information to extract sharp details from all LR multi-view to support the SR of the LR input view. Specifically, the proposed Geometry-Aware Reference Synthesis module in MVSRnet uses geometry information and all multi-view LR images to synthesize pixel-aligned HR reference images. Then, the proposed Dynamic High-Frequency Search network fully exploits the high-frequency textural details in reference images for SR. Extensive experiments on several benchmarks show that our method significantly improves over the state-of-the-art approaches. © 2022, CC BY.

关键词： Geometry

IC-CViT: Inverse-Consistent Convolutional Vision Transformer for Diffeomorphic Image Registration

学校读者我要写书评

暂无评论

IC-CViT: Inverse-Consistent Convolutional Vision Transformer...

International Joint Conference on Neural Networks (IJCNN)

作者： Tao Xu Ting Jiang Xiaoning Li College of Computer Science Sichuan Normal University Chengdu China Visual Computing and Virtual Reality Key Laboratory of Sichuan Province Chengdu China Sichuan 2011 Collaborative Innovation Center for Educational Big Data Chengdu China

Diffeomorphic registration plays a crucial role in medical image analysis due to the invertible and one-to-one mapping transformation. In recent years, with the development of deep learning technology, convolutional neural networks (CNNs) have been a broad focus of research in medical image registration, and CNN-based methods have made great progress. However, the results of most existing methods generally are not necessarily diffeomorphic, generating implausibly bijective mappings between images due to the interpolation and discrete representation. Furthermore, the performances of CNNs may be limited by a lack of precise comprehension of global and long-range cross-image spatial relevance. Vision Transformer (ViT) is capable of enhancing the long-distance information interaction ability to identify the semantically anatomically correspondences of medical images. Compared with CNN, ViT has weak local feature extraction ability due to less inductive bias, especially in small-scale training datasets, meaning that the samples between adjacent pixels cannot be exploited adequately. To address the above challenges, we propose a novel Inverse-Consistent Convolutional Vision Transformer (IC-CViT) network for diffeomorphic image registration. Specifically, image pairs can explicitly conduct bi-directional registration through the predicted deformation filed, generated within the diffeomorphic mappings space and restricted by the proposed inverse consistent loss term. We verify our method on two 3D brain MRI scan datasets including OASIS and LPBA40. Comprehensive results demonstrate that IC-CViT achieves state-of-the-art registration accuracy while maintaining desired diffeomorphic properties.

关键词：

DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Hui Wang, Zheng Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China School of Computer Science Zhejiang University of Technology China

Anomaly detection has garnered extensive applications in real industrial manufacturing due to its remarkable effectiveness and efficiency. However, previous generative-based models have been limited by suboptimal reconstruction quality, hampering their overall performance. We introduce DiffusionAD, a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network. A fundamental enhancement lies in our reformulation of the reconstruction process using a diffusion model into a noise-to-norm paradigm. Here, anomalous regions are perturbed with Gaussian noise and reconstructed as normal, overcoming the limitations of previous models by facilitating anomaly-free restoration. Additionally, we propose a rapid one-step denoising paradigm, significantly faster than the traditional iterative denoising in diffusion models. Furthermore, the introduction of the norm-guided paradigm elevates the accuracy and fidelity of reconstructions. The segmentation sub-network predicts pixel-level anomaly scores using the input image and its anomaly-free restoration. Comprehensive evaluations on four standard and challenging benchmarks reveal that DiffusionAD outperforms current state-of-the-art approaches, demonstrating the effectiveness and broad applicability of the proposed pipeline. Copyright © 2023, The Authors. All rights reserved.

关键词： Anomaly detection

From Hypergraph Energy Functions to Hypergraph Neural Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Yuxin Gan, Quan Qiu, Xipeng Huang, Xuanjing Wipf, David School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Amazon United States Peng Cheng Laboratory China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China

Hypergraphs are a powerful abstraction for representing higher-order interactions between entities of interest. To exploit these relationships in making downstream predictions, a variety of hypergraph neural network architectures have recently been proposed, in large part building upon precursors from the more traditional graph neural network (GNN) literature. Somewhat differently, in this paper we begin by presenting an expressive family of parameterized, hypergraph-regularized energy functions. We then demonstrate how minimizers of these energies effectively serve as node embeddings that, when paired with a parameterized classifier, can be trained end-to-end via a supervised bilevel optimization process. Later, we draw parallels between the implicit architecture of the predictive models emerging from the proposed bilevel hypergraph optimization, and existing GNN architectures in common use. Empirically, we demonstrate state-of-the-art results on various hypergraph node classification benchmarks. Code is available at https://***/yxzwang/PhenomNN. Copyright © 2023, The Authors. All rights reserved.

关键词： Graph neural networks

A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ye, Junjie Chen, Xuanting Xu, Nuo Zu, Can Shao, Zekai Liu, Shichun Cui, Yuhan Zhou, Zeyang Gong, Chao Shen, Yang Zhou, Jie Chen, Siming Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University Shanghai China School of Data Science Fudan University Shanghai China Institute of Modern Languages and Linguistics Fudan University Shanghai China Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University China

GPT series models, such as GPT-3, CodeX, InstructGPT, ChatGPT, and so on, have gained considerable attention due to their exceptional natural language processing capabilities. However, despite the abundance of research on the difference in capabilities between GPT series models and fine-tuned models, there has been limited attention given to the evolution of GPT series models’ capabilities over time. To conduct a comprehensive analysis of the capabilities of GPT series models, we select six representative models, comprising two GPT-3 series models (i.e., davinci and text-davinci-001) and four GPT-3.5 series models (i.e., code-davinci-002, text-davinci-002, text-davinci-003, and gpt-3.5-turbo). We evaluate their performance on nine natural language understanding (NLU) tasks using 21 datasets. In particular, we compare the performance and robustness of different models for each task under zero-shot and few-shot scenarios. Our extensive experiments reveal that the overall ability of GPT series models on NLU tasks does not increase gradually as the models evolve, especially with the introduction of the RLHF training strategy. While this strategy enhances the models’ ability to generate human-like responses, it also compromises their ability to solve some tasks. Furthermore, our findings indicate that there is still room for improvement in areas such as model robustness. Copyright © 2023, The Authors. All rights reserved.

关键词： Natural language processing systems

On the Effects of Self-supervision and Contrastive Alignment in Deep Multi-view Clustering

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Trosten, Daniel J. Løkse, Sigurd Jenssen, Robert Kampffmeyer, Michael C. Department of Physics and Technology UiT The Arctic University of Norway Norway UiT Machine Learning group Visual Intelligence Centre Norway Norwegian Computing Center Norway Department of Computer Science University of Copenhagen Denmark Pioneer Centre for AI

Self-supervised learning is a central component in recent approaches to deep multi-view clustering (MVC). However, we find large variations in the development of self-supervision-based methods for deep MVC, potentially slowing the progress of the field. To address this, we present DeepMVC, a unified framework for deep MVC that includes many recent methods as instances. We leverage our framework to make key observations about the effect of self-supervision, and in particular, drawbacks of aligning representations with contrastive learning. Further, we prove that contrastive alignment can negatively influence cluster separability, and that this effect becomes worse when the number of views increases. Motivated by our findings, we develop several new DeepMVC instances with new forms of self-supervision. We conduct extensive experiments and find that (i) in line with our theoretical findings, contrastive alignments decreases performance on datasets with many views;(ii) all methods benefit from some form of self-supervision;and (iii) our new instances outperform previous methods on several datasets. Based on our results, we suggest several promising directions for future research. To enhance the openness of the field, we provide an open-source implementation of DeepMVC, including recent models and our new instances. Our implementation includes a consistent evaluation protocol, facilitating fair and accurate evaluation of methods and components1 Copyright © 2023, The Authors. All rights reserved.

关键词： Alignment

Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation

学校读者我要写书评

暂无评论

IEEE Transactions on Pattern Analysis and Machine Intelligence 2025年第7期47卷 5520-5537页

作者： Jian Liu Wei Sun Hui Yang Pengchao Deng Chongpei Liu Nicu Sebe Hossein Rahmani Ajmal Mian National Engineering Research Center for Robot Visual Perception and Control Technology College of Electrical and Information Engineering School of Robotics and the State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body Hunan University Changsha China Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Xi’an China Department of Information Engineering and Computer Science University of Trento Trento Italy School of Computing and Communications Lancaster University Lancaster U.K. Department of Computer Science The University of Western Australia Crawley WA Australia

Nine-degrees-of-freedom (9-DoF) object pose and size estimation is crucial for enabling augmented reality and robotic manipulation. Category-level methods have received extensive research attention due to their potential for generalization to intra-class unknown objects. However, these methods require manual collection and labeling of large-scale real-world training data. To address this problem, we introduce a diffusion-based paradigm for domain-generalized category-level 9-DoF object pose estimation. Our motivation is to leverage the latent generalization ability of the diffusion model to address the domain generalization challenge in object pose estimation. This entails training the model exclusively on rendered synthetic data to achieve generalization to real-world scenes. We propose an effective diffusion model to redefine 9-DoF object pose estimation from a generative perspective. Our model does not require any 3D shape priors during training or inference. By employing the Denoising Diffusion Implicit Model, we demonstrate that the reverse diffusion process can be executed in as few as 3 steps, achieving near real-time performance. Finally, we design a robotic grasping system comprising both hardware and software components. Through comprehensive experiments on two benchmark datasets and the real-world robotic system, we show that our method achieves state-of-the-art domain generalization performance.

关键词： Shape Solid modeling Three-dimensional displays Pose estimation Training Robots Grasping Diffusion models Real-time systems Point cloud compression

ADAMANIP: ADAPTIVE ARTICULATED OBJECT MANIPULATION ENVIRONMENTS AND POLICY LEARNING

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Yuanfei Zhang, Xiaojie Wu, Ruihai Li, Yu Shen, Yan Wu, Mingdong He, Zhaofeng Wang, Yizhou Dong, Hao Center on Frontiers of Computing Studies School of Computer Science Peking University China Beijing University of Posts and Telecommunications China Inst. for Artificial Intelligence Peking University China Nat’l Eng. Research Center of Visual Technology Peking University China State Key Laboratory of General Artificial Intelligence Peking University China

Articulated object manipulation is a critical capability for robots to perform various tasks in real-world scenarios. Composed of multiple parts connected by joints, articulated objects are endowed with diverse functional mechanisms through complex relative motions. For example, a safe consists of a door, a handle, and a lock, where the door can only be opened when the latch is unlocked. The internal structure, such as the state of a lock or joint angle constraints, cannot be directly observed from visual observation. Consequently, successful manipulation of these objects requires adaptive adjustment based on trial and error rather than a one-time visual inference. However, previous datasets and simulation environments for articulated objects have primarily focused on simple manipulation mechanisms where the complete manipulation process can be inferred from the object’s appearance. To enhance the diversity and complexity of adaptive manipulation mechanisms, we build a novel articulated object manipulation environment and equip it with 9 categories of objects. Based on the environment and objects, we further propose an adaptive demonstration collection and 3D visual diffusion-based imitation learning pipeline that learns the adaptive manipulation policy. The effectiveness of our designs and proposed method is validated through both simulation and real-world experiments. Our project page is available at: https://*** Copyright © 2025, The Authors. All rights reserved.

关键词： Three dimensional computer graphics