检索结果-内蒙古大学图书馆

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Ziqi Cai Kaiwen Jiang Shu-Yu Chen Yu-Kun Lai Hongbo Fu Boxin Shi Lin Gao Beijing Key Laboratory of Mobile Computing and Pervasive Device Institute of Computing Technology Chinese Academy of Sciences Beijing Jiaotong University University of California San Diego Cardiff University City University of Hong Kong The Hong Kong University of Science and Technology National Key Laboratory for Multimedia Information Processing School of Computer Science Peking University National Engineering Research Center of Visual Technology School of Computer Science Peking University University of Chinese Academy of Sciences

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Synthesizing realistic videos of talking faces under custom lighting conditions and viewing angles benefits various downstream applications like video conferencing. However, most existing relighting methods are either time-consuming or unable to adjust the viewpoints. In this paper, we present the first real-time 3D-aware method for relighting in-the-wild videos of talking faces based on Neural Radiance Fields (NeRF). Given an input portrait video, our method can synthesize talking faces under both novel views and novel lighting conditions with a photo-realistic and disentangled 3D representation. Specifically, we infer an albedo tri-plane, as well as a shading tri-plane based on a desired lighting condition for each video frame with fast dual-encoders. We also leverage a temporal consistency network to ensure smooth transitions and reduce flickering artifacts. Our method runs at 32.98 fps on consumer-level hardware and achieves state-of-the-art results in terms of reconstruction quality, lighting error, lighting instability, temporal consistency and inference speed. We demonstrate the effectiveness and interactivity of our method on various portrait videos with diverse lighting and viewing conditions.

关键词： computer vision Three-dimensional displays Lighting Neural radiance field Real-time systems Hardware Pattern recognition

来源：评论

学校读者我要写书评

暂无评论

Rethinking the Functionality of Latent Representation: A Logarithmic Rate-Distortion Model for Learned Image Compression

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Ge, Ziqing Huang, Zhimeng Jia, Chuanmin Ma, Siwei Gao, Wen School of Computer Science Peking University State Key Laboratory of Multimedia Information Processing Beijing100871 China Institute of Computing Technology Beijing100190 China University of Chinese Academy of Sciences Beijing100190 China Peking University National Engineering Research Center of Visual Technology School of Computer Science Beijing100871 China Peking University Wangxuan Institute of Computer Technology Beijing100871 China Peking University State Key Laboratory of Multimedia Information Processing School of Computer Science Beijing100871 China Chinese Academy of Sciences Beijing100190 China

End-to-end optimized Learned Image Compression (LIC) has demonstrated remarkable performance in terms of Rate-Distortion (R-D) efficiency. However, the R-D characteristics of LIC codecs remain underexplored. Previous research has attempted to investigate the R-D behavior through numerical and statistical approaches, but these methods often provide only empirical results, lacking theoretical insights. In this work, we introduce a novel methodology for studying the R-D characteristics of LIC. By rethinking the LIC paradigm from a fresh perspective, we propose a plug-and-play module, the Latent-domain Auto-Encoder (LAE). This innovative approach not only naturally leads to Variable Bit-Rate (VBR) compression, but also allows for a theoretical modeling of the R-D behavior of LIC codecs. Our findings reveal that the bit-rate is the logarithmic sum of the neurons nλ in our designed network's last layer, plus a constant C introduced by image content, formally expressed as Rλ = Σ log nλ + C. This insight is pivotal, as it underscores how the bit-rate can be systematically derived from the latent representations. Further analysis demonstrates that our proposed R-λ model enables effective rate control for learned image codecs, enhancing their adaptability and accuracy. Experimental results validate that our VBR method surpasses fixed-rate coding by 2.9% in terms of BD-rate. Additionally, the proposed R-λ model exhibits superior rate control performance, suggesting that it not only elucidates the underlying R-D characteristics of LIC but also significantly enhances its practical deployment in real-world applications. © 1991-2012 IEEE.

关键词： Image coding

来源：评论

学校读者我要写书评

暂无评论

Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-Shot Learning with Hyperspherical Embeddings

Hubs and Hyperspheres: Reducing Hubness and Improving Transd...

引用

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Daniel J. Trosten Rwiddhi Chakraborty Sigurd Løksc Kristoffer Knutsen Wickstrøm Robert Jenssen Michael C. Kampffmeyer Department of Physics and Technology UiT The Arctic University of Norway UiT Machine Learning group (machine-learning.uit. no) and Visual Intelligence Centre (visual-intelligence.no) Norwegian Computing Center Department of Computer Science University of Copenhagen Pioneer Centre for AI (aicentre.dk)

Distance-based classification is frequently used in transductive few-shot learning (FSL). However, due to the high-dimensionality of image representations, FSL classifiers are prone to suffer from the hubness problem, where a few points (hubs) occur frequently in multiple nearest neighbour lists of other points. Hubness negatively impacts distance-based classification when hubs from one class appear often among the nearest neighbors of points from another class, degrading the classifier's performance. To address the hubness problem in FSL, we first prove that hubness can be eliminated by distributing representations uniformly on the hypersphere. We then propose two new approaches to embed representations on the hypersphere, which we prove optimize a tradeoff between uniformity and local similarity preservation - reducing hubness while retaining class structure. Our experiments show that the proposed methods reduce hubness, and significantly improves transductive FSL accuracy for a wide range of classifiers 1 1 Code available at https://***/uitml/noHub..

关键词：

来源：评论

学校读者我要写书评

暂无评论

Learning Parallax Transformer Network for Stereo Image JPEG Artifacts Removal

arXiv

引用

arXiv 2022年

作者： Jiang, Xuhao Tan, Weimin Cheng, Ri Zhou, Shili Yan, Bo School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai China

Under stereo settings, the performance of image JPEG artifacts removal can be further improved by exploiting the additional information provided by a second view. However, incorporating this information for stereo image JPEG artifacts removal is a huge challenge, since the existing compression artifacts make pixel-level view alignment difficult. In this paper, we propose a novel parallax transformer network (PTNet) to integrate the information from stereo image pairs for stereo image JPEG artifacts removal. Specifically, a well-designed symmetric bi-directional parallax transformer module is proposed to match features with similar textures between different views instead of pixel-level view alignment. Due to the issues of occlusions and boundaries, a confidence-based cross-view fusion module is proposed to achieve better feature fusion for both views, where the cross-view features are weighted with confidence maps. Especially, we adopt a coarse-to-fine design for the cross-view interaction, leading to better performance. Comprehensive experimental results demonstrate that our PTNet can effectively remove compression artifacts and achieves superior performance than other testing state-of-the-art methods. © 2022, CC BY.

关键词： Textures

来源：评论

学校读者我要写书评

暂无评论

Geometry-Aware Reference Synthesis for Multi-View Image Super-Resolution

arXiv

引用

arXiv 2022年

作者： Cheng, Ri Sun, Yuqi Yan, Bo Tan, Weimin Ma, Chenxi School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai China

Recent multi-view multimedia applications struggle between high-resolution (HR) visual experience and storage or bandwidth constraints. Therefore, this paper proposes a Multi-View Image Super-Resolution (MVISR) task. It aims to increase the resolution of multiview images captured from the same scene. One solution is to apply image or video super-resolution (SR) methods to reconstruct HR results from the low-resolution (LR) input view. However, these methods cannot handle large-angle transformations between views and leverage information in all multi-view images. To address these problems, we propose the MVSRnet, which uses geometry information to extract sharp details from all LR multi-view to support the SR of the LR input view. Specifically, the proposed Geometry-Aware Reference Synthesis module in MVSRnet uses geometry information and all multi-view LR images to synthesize pixel-aligned HR reference images. Then, the proposed Dynamic High-Frequency Search network fully exploits the high-frequency textural details in reference images for SR. Extensive experiments on several benchmarks show that our method significantly improves over the state-of-the-art approaches. © 2022, CC BY.

关键词： Geometry

来源：评论

学校读者我要写书评

暂无评论

IC-CViT: Inverse-Consistent Convolutional Vision Transformer for Diffeomorphic Image Registration

IC-CViT: Inverse-Consistent Convolutional Vision Transformer...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Tao Xu Ting Jiang Xiaoning Li College of Computer Science Sichuan Normal University Chengdu China Visual Computing and Virtual Reality Key Laboratory of Sichuan Province Chengdu China Sichuan 2011 Collaborative Innovation Center for Educational Big Data Chengdu China

Diffeomorphic registration plays a crucial role in medical image analysis due to the invertible and one-to-one mapping transformation. In recent years, with the development of deep learning technology, convolutional neural networks (CNNs) have been a broad focus of research in medical image registration, and CNN-based methods have made great progress. However, the results of most existing methods generally are not necessarily diffeomorphic, generating implausibly bijective mappings between images due to the interpolation and discrete representation. Furthermore, the performances of CNNs may be limited by a lack of precise comprehension of global and long-range cross-image spatial relevance. Vision Transformer (ViT) is capable of enhancing the long-distance information interaction ability to identify the semantically anatomically correspondences of medical images. Compared with CNN, ViT has weak local feature extraction ability due to less inductive bias, especially in small-scale training datasets, meaning that the samples between adjacent pixels cannot be exploited adequately. To address the above challenges, we propose a novel Inverse-Consistent Convolutional Vision Transformer (IC-CViT) network for diffeomorphic image registration. Specifically, image pairs can explicitly conduct bi-directional registration through the predicted deformation filed, generated within the diffeomorphic mappings space and restricted by the proposed inverse consistent loss term. We verify our method on two 3D brain MRI scan datasets including OASIS and LPBA40. Comprehensive results demonstrate that IC-CViT achieves state-of-the-art registration accuracy while maintaining desired diffeomorphic properties.

关键词：

来源：评论

学校读者我要写书评

暂无评论

DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection

arXiv

引用

arXiv 2023年

作者： Zhang, Hui Wang, Zheng Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China School of Computer Science Zhejiang University of Technology China

Anomaly detection has garnered extensive applications in real industrial manufacturing due to its remarkable effectiveness and efficiency. However, previous generative-based models have been limited by suboptimal reconstruction quality, hampering their overall performance. We introduce DiffusionAD, a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network. A fundamental enhancement lies in our reformulation of the reconstruction process using a diffusion model into a noise-to-norm paradigm. Here, anomalous regions are perturbed with Gaussian noise and reconstructed as normal, overcoming the limitations of previous models by facilitating anomaly-free restoration. Additionally, we propose a rapid one-step denoising paradigm, significantly faster than the traditional iterative denoising in diffusion models. Furthermore, the introduction of the norm-guided paradigm elevates the accuracy and fidelity of reconstructions. The segmentation sub-network predicts pixel-level anomaly scores using the input image and its anomaly-free restoration. Comprehensive evaluations on four standard and challenging benchmarks reveal that DiffusionAD outperforms current state-of-the-art approaches, demonstrating the effectiveness and broad applicability of the proposed pipeline. Copyright © 2023, The Authors. All rights reserved.

关键词： Anomaly detection

来源：评论

学校读者我要写书评

暂无评论

From Hypergraph Energy Functions to Hypergraph Neural Networks

arXiv

引用

arXiv 2023年

作者： Wang, Yuxin Gan, Quan Qiu, Xipeng Huang, Xuanjing Wipf, David School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Amazon United States Peng Cheng Laboratory China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China

Hypergraphs are a powerful abstraction for representing higher-order interactions between entities of interest. To exploit these relationships in making downstream predictions, a variety of hypergraph neural network architectures have recently been proposed, in large part building upon precursors from the more traditional graph neural network (GNN) literature. Somewhat differently, in this paper we begin by presenting an expressive family of parameterized, hypergraph-regularized energy functions. We then demonstrate how minimizers of these energies effectively serve as node embeddings that, when paired with a parameterized classifier, can be trained end-to-end via a supervised bilevel optimization process. Later, we draw parallels between the implicit architecture of the predictive models emerging from the proposed bilevel hypergraph optimization, and existing GNN architectures in common use. Empirically, we demonstrate state-of-the-art results on various hypergraph node classification benchmarks. Code is available at https://***/yxzwang/PhenomNN. Copyright © 2023, The Authors. All rights reserved.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models

arXiv

引用

arXiv 2023年

作者： Ye, Junjie Chen, Xuanting Xu, Nuo Zu, Can Shao, Zekai Liu, Shichun Cui, Yuhan Zhou, Zeyang Gong, Chao Shen, Yang Zhou, Jie Chen, Siming Gui, Tao Zhang, Qi Huang, Xuanjing School of Computer Science Fudan University Shanghai China School of Data Science Fudan University Shanghai China Institute of Modern Languages and Linguistics Fudan University Shanghai China Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University China

GPT series models, such as GPT-3, CodeX, InstructGPT, ChatGPT, and so on, have gained considerable attention due to their exceptional natural language processing capabilities. However, despite the abundance of research on the difference in capabilities between GPT series models and fine-tuned models, there has been limited attention given to the evolution of GPT series models’ capabilities over time. To conduct a comprehensive analysis of the capabilities of GPT series models, we select six representative models, comprising two GPT-3 series models (i.e., davinci and text-davinci-001) and four GPT-3.5 series models (i.e., code-davinci-002, text-davinci-002, text-davinci-003, and gpt-3.5-turbo). We evaluate their performance on nine natural language understanding (NLU) tasks using 21 datasets. In particular, we compare the performance and robustness of different models for each task under zero-shot and few-shot scenarios. Our extensive experiments reveal that the overall ability of GPT series models on NLU tasks does not increase gradually as the models evolve, especially with the introduction of the RLHF training strategy. While this strategy enhances the models’ ability to generate human-like responses, it also compromises their ability to solve some tasks. Furthermore, our findings indicate that there is still room for improvement in areas such as model robustness. Copyright © 2023, The Authors. All rights reserved.

关键词： Natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

On the Effects of Self-supervision and Contrastive Alignment in Deep Multi-view Clustering

arXiv

引用

arXiv 2023年

作者： Trosten, Daniel J. Løkse, Sigurd Jenssen, Robert Kampffmeyer, Michael C. Department of Physics and Technology UiT The Arctic University of Norway Norway UiT Machine Learning group Visual Intelligence Centre Norway Norwegian Computing Center Norway Department of Computer Science University of Copenhagen Denmark Pioneer Centre for AI

Self-supervised learning is a central component in recent approaches to deep multi-view clustering (MVC). However, we find large variations in the development of self-supervision-based methods for deep MVC, potentially slowing the progress of the field. To address this, we present DeepMVC, a unified framework for deep MVC that includes many recent methods as instances. We leverage our framework to make key observations about the effect of self-supervision, and in particular, drawbacks of aligning representations with contrastive learning. Further, we prove that contrastive alignment can negatively influence cluster separability, and that this effect becomes worse when the number of views increases. Motivated by our findings, we develop several new DeepMVC instances with new forms of self-supervision. We conduct extensive experiments and find that (i) in line with our theoretical findings, contrastive alignments decreases performance on datasets with many views;(ii) all methods benefit from some form of self-supervision;and (iii) our new instances outperform previous methods on several datasets. Based on our results, we suggest several promising directions for future research. To enhance the openness of the field, we provide an open-source implementation of DeepMVC, including recent models and our new instances. Our implementation includes a consistent evaluation protocol, facilitating fair and accurate evaluation of methods and components1 Copyright © 2023, The Authors. All rights reserved.

关键词： Alignment

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：