检索结果-内蒙古大学图书馆

MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Shili Jiang, Xuhao Tan, Weimin He, Ruian Yan, Bo School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai China

In recent years, many deep learning-based methods have been proposed to tackle the problem of optical flow estimation and achieved promising results. However, they hardly consider that most videos are compressed and thus ignore the pre-computed information in compressed video streams. Motion vectors, one of the compression information, record the motion of the video frames. They can be directly extracted from the compression code stream without computational cost and serve as a solid prior for optical flow estimation. Therefore, we propose an optical flow model, MVFlow, which uses motion vectors to improve the speed and accuracy of optical flow estimation for compressed videos. In detail, MVFlow includes a key Motion-Vector Converting Module, which ensures that the motion vectors can be transformed into the same domain of optical flow and then be utilized fully by the flow estimation module. Meanwhile, we construct four optical flow datasets for compressed videos containing frames and motion vectors in pairs. The experimental results demonstrate the superiority of our proposed MVFlow, which can reduce the AEPE by 1.09 compared to existing models or save 52% time to achieve similar accuracy to existing models. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical flows

Uncertainty-Guided Spatial Pruning Architecture for Efficient Frame Interpolation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Cheng, Ri Jiang, Xuhao He, Ruian Zhou, Shili Tan, Weimin Yan, Bo School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai China

The video frame interpolation (VFI) model applies the convolution operation to all locations, leading to redundant computations in regions with easy motion. We can use dynamic spatial pruning method to skip redundant computation, but this method cannot properly identify easy regions in VFI tasks without supervision. In this paper, we develop an Uncertainty-Guided Spatial Pruning (UGSP) architecture to skip redundant computation for efficient frame interpolation dynamically. Specifically, pixels with low uncertainty indicate easy regions, where the calculation can be reduced without bringing undesirable visual results. Therefore, we utilize uncertainty-generated mask labels to guide our UGSP in properly locating the easy region. Furthermore, we propose a self-contrast training strategy that leverages an auxiliary non-pruning branch to improve the performance of our UGSP. Extensive experiments show that UGSP maintains performance but reduces FLOPs by 34%/52%/30% compared to baseline without pruning on Vimeo90K/UCF101/MiddleBury datasets. In addition, our method achieves state-of-the-art performance with lower FLOPs on multiple benchmarks. Copyright © 2023, The Authors. All rights reserved.

关键词： Benchmarking

Friedkin-Johnsen Model for Opinion Dynamics on Signed Graphs

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Xiaotian Sun, Haoxin Xu, Wanyue Li, Wei Zhang, Zhongzhi Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Shanghai200433 China The Academy for Engineering and Technology Fudan University Shanghai200433 China

A signed graph offers richer information than an unsigned graph, since it describes both collaborative and competitive relationships in social networks. In this paper, we study opinion dynamics on a signed graph, based on the Friedkin-Johnsen model. We first interpret the equilibrium opinion in terms of a defined random walk on an augmented signed graph, by representing the equilibrium opinion of every node as a combination of all nodes’ internal opinions, with the coefficient of the internal opinion for each node being the difference of two absorbing probabilities. We then quantify some relevant social phenomena and express them in terms of the 2 norms of vectors. We also design a nearly-linear time signed Laplacian solver for assessing these quantities, by establishing a connection between the absorbing probability of random walks on a signed graph and that on an associated unsigned graph. We further study the opinion optimization problem by changing the initial opinions of a fixed number of nodes, which can be optimally solved in cubic time. We provide a nearly-linear time algorithm with error guarantee to approximately solve the problem. Finally, we execute extensive experiments on sixteen real-life signed networks, which show that both of our algorithms are effective and efficient, and are scalable to massive graphs with over 20 million nodes. Copyright © 2024, The Authors. All rights reserved.

关键词： Clustering algorithms

Fine-Grained Blind Face Inpainting with 3D Face Component Disentanglement

学校读者我要写书评

暂无评论

Fine-Grained Blind Face Inpainting with 3D Face Component Di...

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Yu Bai Ruian He Weimin Tan Bo Yan Yangle Lin School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai China

Inpainting is a task to restore occlusion or other corruption on images. However, previous works require mask of the occluded area to restore the occluded image, which is inconvenient for application. Blind face inpainting aims to automatically restore the occluded face without position information of the corrupt region. In this paper, we propose a novel fine-grained blind face inpainting framework, combining 3D face components disentanglement with generative network. Canonical face texture and shape disentangled by unsupervised 3D face model is restored separately to get occlusion-free rendered result. Finally, the pixel-to-pixel generative module utilize the occlusion image and the coarse de-occlusion face to get refined inpainted result. We also build up a new dataset called CelebO-3D which consists of occluded face images synthesized with 3D occlusion and rendered by 3D face model. Extensive experiments show that the proposed method is effective and robust in face blind inpainting both in synthesized and real images. Extensive evaluations and comparison with previous methods also show our superior effectiveness and lightweight architecture.

关键词： Solid modeling Three-dimensional displays Shape Speech enhancement Signal processing Acoustics Image restoration

Self-supervised Learning of Orc-Bert Augmentor for Recognizing Few-Shot Oracle Characters 15th

学校读者我要写书评

暂无评论

Self-supervised Learning of Orc-Bert Augmentor for Recognizi...

15th Asian Conference on computer Vision, ACCV 2020

作者： Han, Wenhui Ren, Xinlin Lin, Hangyu Fu, Yanwei Xue, Xiangyang School of Data Science Computer Science and MOE Frontiers Center for Brain Science Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai China

ISBN: (纸本)9783030695439

This paper studies the recognition of oracle character, the earliest known hieroglyphs in China. Essentially, oracle character recognition suffers from the problem of data limitation and imbalance. Recognizing the oracle characters of extremely limited samples, naturally, should be taken as the few-shot learning task. Different from the standard few-shot learning setting, our model has only access to large-scale unlabeled source Chinese characters and few labeled oracle characters. In such a setting, meta-based or metric-based few-shot methods are failed to be efficiently trained on source unlabeled data;and thus the only possible methodologies are self-supervised learning and data augmentation. Unfortunately, the conventional geometric augmentation always performs the same global transformations to all samples in pixel format, without considering the diversity of each part within a sample. Moreover, to the best of our knowledge, there is no effective self-supervised learning method for few-shot learning. To this end, this paper integrates the idea of self-supervised learning in data augmentation. And we propose a novel data augmentation approach, named Orc-Bert Augmentor pre-trained by self-supervised learning, for few-shot oracle character recognition. Specifically, Orc-Bert Augmentor leverages a self-supervised BERT model pre-trained on large unlabeled Chinese characters datasets to generate sample-wise augmented samples. Given a masked input in vector format, Orc-Bert Augmentor can recover it and then output a pixel format image as augmented data. Different mask proportion brings diverse reconstructed output. Concatenated with Gaussian noise, the model further performs point-wise displacement to improve diversity. Experimentally, we collect two large-scale datasets of oracle characters and other Chinese ancient characters for few-shot oracle character recognition and Orc-Bert Augmentor pre-training. Extensive experiments on few-shot learning demonstrate the effe

关键词： Character recognition

Improving Drum Source Separation with Temporal-Frequency Statistical Descriptors

学校读者我要写书评

暂无评论

Improving Drum Source Separation with Temporal-Frequency Sta...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Si Li Jiaxing Liu Peilin Li Dichucheng Li Xinlu Liu Yongwei Gao Wei Li School of Computer Science and Technology Fudan University China School of Statistics and Information Shanghai University of International Business and Economics China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China

ISBN: (数字)9798350390155

ISBN: (纸本)9798350390162

Drum Source Separation (DSS) aims to separate drum mixtures into individual drum sounds, such as kick and snare. Deep neural network methods have been successfully applied for source separation. However, due to the limited size of existing datasets and the strongly overlap of drums in frequency and time, these methods still have certain shortcomings. To address these challenges, we construct a large drum sound dataset and propose a novel training objective to improve performance of DSS task. The training objective leverages three temporal-frequency statistical descriptors (spectral centroid, spectral spread, and spectral flux) to separate drum sources. Our experimental results demonstrate that our method can make a SDR improvement of 0.98 dB on UNet and 1.07 dB on MERT. Furthermore, our method achieves consistent improvements in low-resource and cross-dataset scenarios. Our code and dataset are available at https://***/150042/Drum-Separation-TF.

关键词： Training Deep learning Adaptation models Time-frequency analysis Visualization Tracking loops Source separation

SEMANTIC LATENT DECOMPOSITION WITH NORMALIZING FLOWS FOR FACE EDITING

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Binglei Huang, Zhizhong Shan, Hongming Zhang, Junping Shanghai Key Lab of Intelligent Information Processing School of Computer Science China Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China

Navigating in the latent space of StyleGAN has shown effectiveness for face editing. However, the resulting methods usually encounter challenges in complicated navigation due to the entanglement among different attributes in the latent space. To address this issue, this paper proposes a novel framework, termed SDFlow, with a semantic decomposition in original latent space using continuous conditional normalizing flows. Specifically, SDFlow decomposes the original latent code into different irrelevant variables by jointly optimizing two components: (i) a semantic encoder to estimate semantic variables from input faces and (ii) a flow-based transformation module to map the latent code into a semantic-irrelevant variable in Gaussian distribution, conditioned on the learned semantic variables. To eliminate the entanglement between variables, we employ a disentangled learning strategy under a mutual information framework, thereby providing precise manipulation controls. Experimental results demonstrate that SDFlow outperforms existing state-of-the-art face editing methods both qualitatively and quantitatively. The source code is made available at https://***/phil329/SDFlow. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhu, Qin Huang, Fei Peng, Runyu Lu, Keming Yu, Bowen Cheng, Qinyuan Qiu, Xipeng Huang, Xuanjing Lin, Junyang Qwen Team Alibaba Group China School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China

While logical reasoning evaluation of Large Language Models (LLMs) has attracted significant attention, existing benchmarks predominantly rely on multiple-choice formats that are vulnerable to random guessing, leading to overestimated performance and substantial performance fluctuations. To obtain more accurate assessments of models’ reasoning capabilities, we propose an automated method for synthesizing open-ended logic puzzles, and use it to develop a bilingual benchmark, AutoLogi. Our approach features program-based verification and controllable difficulty levels, enabling more reliable evaluation that better distinguishes models’ reasoning abilities. Extensive evaluation of eight modern LLMs shows that AutoLogi can better reflect true model capabilities, with performance scores spanning from 35% to 73% compared to the narrower range of 21% to 37% on the source multiple-choice dataset. Beyond benchmark creation, this synthesis method can generate high-quality training data by incorporating program verifiers into the rejection sampling process, enabling systematic enhancement of LLMs’ reasoning capabilities across diverse datasets. © 2025, CC BY-SA.

关键词： computer circuits

CROSS-HEAD SUPERVISION FOR CROWD COUNTING WITH NOISY ANNOTATIONS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Dai, Mingliang Huang, Zhizhong Gao, Jiaqi Shan, Hongming Zhang, Junping Shanghai Key Lab of Intelligent Information Processing School of Computer Science China Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China

Noisy annotations such as missing annotations and location shifts often exist in crowd counting datasets due to multi-scale head sizes, high occlusion, etc. These noisy annotations severely affect the model training, especially for density map-based methods. To alleviate the negative impact of noisy annotations, we propose a novel crowd counting model with one convolution head and one transformer head, in which these two heads can supervise each other in noisy areas, called Cross-Head Supervision. The resultant model, CHS-Net, can synergize different types of inductive biases for better counting. In addition, we develop a progressive cross-head supervision learning strategy to stabilize the training process and provide more reliable supervision. Extensive experimental results on shanghaiTech and QNRF datasets demonstrate superior performance over state-of-the-art methods. Code is available at https://***/RaccoonDML/CHSNet. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning