检索结果-内蒙古大学图书馆

GenRec: unifying video generation and recognition with diffusion models 24

学校读者我要写书评

暂无评论

GenRec: unifying video generation and recognition with diffu...

Proceedings of the 38th International Conference on Neural information processing Systems

作者： Zejia Weng Xitong Yang Zhen Xing Zuxuan Wu Yu-Gang Jiang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University and Shanghai Collaborative Innovation Center of Intelligent Visual Computing Department of Computer Science University of Maryland

ISBN: (纸本)9798331314385

Video diffusion models are able to generate high-quality videos by learning strong spatial-temporal priors on large-scale datasets. In this paper, we aim to investigate whether such priors derived from a generative process are suitable for video recognition, and eventually joint optimization of generation and recognition. Building upon Stable Video Diffusion, we introduce GenRec, the first unified framework trained with a random-frame conditioning process so as to learn generalized spatial-temporal representations. The resulting framework can naturally supports generation and recognition, and more importantly is robust even when visual inputs contain limited information. Extensive experiments demonstrate the efficacy of GenRec for both recognition and generation. In particular, GenRec achieves competitive recognition performance, offering 75.8% and 87.2% accuracy on SSV2 and K400, respectively. GenRec also performs the best on class-conditioned image-to-video generation, achieving 46.5 and 49.3 FVD scores on SSV2 and EK-100 datasets. Furthermore, GenRec demonstrates extraordinary robustness in scenarios that only limited frames can be observed. Code will be available at https://***/wengzejia1/GenRec.

关键词：

FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network 39

学校读者我要写书评

暂无评论

FaceA-Net: Facial Attribute-Driven ID Preserving Image Gener...

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Wang, Jiayu Yu, Yue Chen, Jingjing Dai, Qi Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Microsoft Research Asia China

ISBN: (纸本)157735897X

Recent advances in diffusion-based generative models have demonstrated superior performance in subject-driven image generation. Identity (ID) preserving image generation, as a subtask of subject-driven image generation, aims to generate customized images for specific human identity and has broad application potential. However, this task remains challenging due to the requirement for high ID fidelity and precise detail preservation. Additionally, generating high-quality context presents another challenge, as existing methods struggle to achieve both high ID fidelity and satisfactory context simultaneously. To address the issues of insufficient ID fidelity, we introduce a simple yet effective test-time fine-tuning approach. Specifically, we propose an attribute-driven training method that establishes global-level and local-level tasks to learn the global face feature and fine-grained attribute features, respectively. Furthermore, we introduce a novel ID-context decoupling framework that decouples image context generation from human ID generation, ensuring the quality of contextual content as well as facilitating the learning of ID information. Through extensive experiments, we demonstrate the effectiveness of the proposed method and showcase its capabilities across various applications. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词：

Spk2ImgMamba: Spiking Camera Image Reconstruction with Multi-Scale State Space Models

学校读者我要写书评

暂无评论

Spk2ImgMamba: Spiking Camera Image Reconstruction with Multi...

IEEE Workshop on Applications of computer Vision (WACV)

作者： Jiaoyang Yin Bin Fan Chao Xu Tiejun Huang Boxin Shi State Key Lab of Multimedia Info. Processing School of Computer Science Peking University Nat'l Eng. Research Ctr. of Visual Technology School of Computer Science Peking University Nat'l Key Lab of General AI School of Intelligence Science and Technology Peking University

ISBN: (数字)9798331510831

ISBN: (纸本)9798331510848

As a bio-inspired vision sensor, the spiking camera has showcased remarkable capability in high-speed imaging with a sampling rate of 40,000 Hz. Reconstructing clear images from continuous spike streams, which is obtained by each photosensor continuously detecting photons and firing them asynchronously, has garnered significant attention. Despite promising results, existing spike-to-image reconstruction methods face challenges in balancing global receptive fields and efficient computation due to the inherent limitations of their backbones. Recently, due to powerful long-range modeling and linear complexity, the state space model (SSM) has emerged as a competitive alternative to CNNs and Transformers. In this paper, we propose a lightweight spike-to-image reconstruction network that harnesses Mamba as the backbone. Our approach sequentially executes three core modules: temporal information integration, spatial feature enhancement, and progressive image reconstruction. The former accumulates cues across diverse temporal windows to explore both long-term and short-term contexts. Subsequently, to model global dependencies while heightening local detail perception, we develop a multi-scale SSM block characterized by multi-scale multi-direction scanning, which effectively boosts spatial feature representations. Finally, intensity images are decoded progressively from the enhanced light-intensity features. Extensive experiments on both synthetic and real-captured data demonstrate that our approach achieves state-of-the-art performance, with only 10% of the network parameters and nearly two orders of magnitude less computational effort. The code will be available at https://***/interstellarH/Spk2ImgMamba.

关键词： Correlation Computational modeling Reconstruction algorithms Vision sensors Transformer cores Streaming media Cameras Transformers Image reconstruction Streams

BDLR:lncRNA identification using ensemble learning

学校读者我要写书评

暂无评论

BIOCELL 2022年第4期46卷 951-960页

作者： LEJUN GONG SHEHAI ZHOU JINGMEI CHEN YONGMIN LI LI ZHANG ZHIHONG GAO Jiangsu Key Lab of Big Data Security&Intelligent Processing School of Computer Science Nanjing University of Posts and TelecommunicationsNanjing210046China Smart Health Big Data Analysis and Location Services Engineering Laboratory of Jiangsu Province Nanjing210046China Zhejiang Engineering Research Center of Intelligent Medicine Wenzhou325035China College of Computer Science and Technology Nanjing Forestry UniversityNanjing210037China

Long non-coding RNAs(lncRNAs)play an important role in many life activities such as epigenetic material regulation,cell cycle regulation,dosage compensation and cell differentiation regulation,and are associated with many human *** are many limitations in identifying and annotating lncRNAs using traditional biological experimental *** the development of high-throughput sequencing technology,it is of great practical significance to identify the lncRNAs from massive RNA sequence data using machine learning *** on the Bagging method and Decision Tree algorithm in ensemble learning,this paper proposes a method of lncRNAs gene sequence identification called *** identification results of this classification method are compared with the identification results of several models including Byes,Support Vector Machine,Logical Regression,Decision Tree and Random *** experimental results show that the lncRNAs identification method named BDLR proposed in this paper has an accuracy of 86.61%in the human test set and 90.34%in the mouse for lncRNAs,which is more than the identification results of the other ***,the proposed method offers a reference for researchers to identify lncRNAs using the ensemble learning.

关键词： lncRNAs High-throughput sequencing Ensemble learning Bagging Decision Tree

Open-Vocabulary Video Relation Extraction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Tian, Wentao Wang, Zheng Fu, Yuqian Chen, Jingjing Cheng, Lechao Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China College of Computer Science and Technology Zhejiang University of Technology China Zhejiang Lab China

A comprehensive understanding of videos is inseparable from describing the action with its contextual action-object interactions. However, many current video understanding tasks prioritize general action classification and overlook the actors and relationships that shape the nature of the action, resulting in a superficial understanding of the action. Motivated by this, we introduce Open-vocabulary Video Relation Extraction (OVRE), a novel task that views action understanding through the lens of action-centric relation triplets. OVRE focuses on pairwise relations that take part in the action and describes these relation triplets with natural languages. Moreover, we curate the Moments-OVRE dataset, which comprises 180K videos with action-centric relation triplets, sourced from a multi-label action classification dataset. With Moments-OVRE, we further propose a crossmodal mapping model to generate relation triplets as a sequence. Finally, we benchmark existing cross-modal generation models on the new task of OVRE. Our code and dataset are available at https://***/Iriya99/OVRE. © 2023, CC BY-NC-ND.

关键词： Extraction

Unlocking Textual and Visual Wisdom: Open-Vocabulary 3D Object Detection Enhanced by Comprehensive Guidance from Text and Image

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jiao, Pengkun Zhao, Na Chen, Jingjing Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Singapore University of Technology and Design Singapore

Open-vocabulary 3D object detection (OV-3DDet) aims to localize and recognize both seen and previously unseen object categories within any new 3D scene. While language and vision foundation models have achieved success in handling various open-vocabulary tasks with abundant training data, OV-3DDet faces a significant challenge due to the limited availability of training data. Although some pioneering efforts have integrated vision-language models (VLM) knowledge into OV-3DDet learning, the full potential of these foundational models has yet to be fully exploited. In this paper, we unlock the textual and visual wisdom to tackle the open-vocabulary 3D detection task by leveraging the language and vision foundation models. We leverage a vision foundation model to provide image-wise guidance for discovering novel classes in 3D scenes. Specifically, we utilize a object detection vision foundation model to enable the zero-shot discovery of objects in images, which serves as the initial seeds and filtering guidance to identify novel 3D objects. Additionally, to align the 3D space with the powerful vision-language space, we introduce a hierarchical alignment approach, where the 3D feature space is aligned with the vision-language feature space using a pre-trained VLM at the instance, category, and scene levels. Through extensive experimentation, we demonstrate significant improvements in accuracy and generalization, highlighting the potential of foundation models in advancing open-vocabulary 3D object detection in real-world scenarios. © 2024, CC BY.

关键词： Object detection

(Image Present) ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Zhihao Jiang, Haoran Chen, Haoran Cao, Yixin Qiu, Xipeng Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China School of Computer Science Fudan University China

(Image Present) Multimodal large language models have unlocked new possibilities for various multimodal tasks. However, their potential in image manipulation detection remains unexplored. When directly applied to the IMD task, M-LLMs often produce reasoning texts that suffer from hallucinations and overthinking. To address this, in this work, we propose ForgerySleuth, which leverages M-LLMs to perform comprehensive clue fusion and generate segmentation outputs indicating specific regions that are tampered with. Moreover, we construct the ForgeryAnalysis dataset through the Chain-of-Clues prompt, which includes analysis and reasoning text to upgrade the image manipulation detection task. A data engine is also introduced to build a larger-scale dataset for the pre-training phase. Our extensive experiments demonstrate the effectiveness of ForgeryAnalysis and show that ForgerySleuth significantly outperforms existing methods in generalization, robustness, and explainability. The resource is available at https://***/sunzhihao18/ForgerySleuth. Copyright © 2024, The Authors. All rights reserved.

关键词： Large datasets

GenRec: Unifying Video Generation and Recognition with Diffusion Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Weng, Zejia Yang, Xitong Xing, Zhen Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Department of Computer Science University of Maryland United States

关键词： Spatio-temporal data

Semi-Supervised Clustering Framework for Fine-grained Scene Graph Generation 39

学校读者我要写书评

暂无评论

Semi-Supervised Clustering Framework for Fine-grained Scene ...

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Yang, Jiarui Wang, Chuan Zhang, Jun Wu, Shuyi Jinjing, Zhao Liu, Zeming Yang, Liang Shanghai Key Lab of Intell. Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Institute of Information Engineering CAS China School of Computer Science and Technology Beijing JiaoTong University China Guangdong Provincial Key Lab of Intell. Info. Processing & Shenzhen Key Lab of Media Security Shenzhen University China Information Research Center of Military Science PLA Academy of Military Science China National Key Laboratory of Science and Technology on Information System Security China School of Computer Science and Engineering Beihang University China School of Artificial Intelligence Hebei University of Technology China

ISBN: (纸本)157735897X

Scene Graph Generation (SGG) aims to detect all objects and identify their pairwise relationships existing in the scene. Considering the substantial human labor costs, existing scene graph annotations are often sparse and biased, which result in confusion training with low-frequency predicates. In this work, we design a Semi-Supervised Clustering framework for Scene Graph Generation (SSC-SGG) that uses the sparse labeled data to guide the generation of effective pseudo-labels from unlabeled object pairs, thus enriching the labeled sample space, especially for low-frequency interaction samples. We approach from the perspective of clustering, reducing the problem of confirmation bias in a self-training manner. Specifically, we first enhance the model’s robustness to feature extraction via prototype-based clustering, aggregating different relationship augmented features onto the same prototype. Secondly, we design a dynamic pseudo-label assignment algorithm based on a mini-batch, which adjusts the detection sensitivity to different frequency samples from the historical assignment. Finally, we conduct joint training on the pseudo-labels and the labeled data. We conduct experiments on various SGG models and achieve substantial overall performance improvements, demonstrating the effectiveness of SSC-SGG. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词：