检索结果-内蒙古大学图书馆

A Recipe for Scaling up Text-to-Video Generation with Text-free Videos

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Xiang Zhang, Shiwei Yuan, Hangjie Qing, Zhiwu Gong, Biao Zhang, Yingya Shen, Yujun Gao, Changxin Sang, Nong Key Laboratory of Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology China Alibaba Group China Zhejiang University China Ant Group China

Diffusion-based text-to-video generation has witnessed impressive progress in the past year yet still falls behind text-to-image generation. One of the key reasons is the limited scale of publicly available data (e.g., 10M video-text pairs in WebVid10M vs. 5B image-text pairs in LAION), considering the high cost of video captioning. Instead, it could be far easier to collect unlabeled clips from video platforms like YouTube. Motivated by this, we come up with a novel text-to-video generation framework, termed TF-T2V, which can directly learn with text-free videos. The rationale behind is to separate the process of text decoding from that of temporal modeling. To this end, we employ a content branch and a motion branch, which are jointly optimized with weights shared. Following such a pipeline, we study the effect of doubling the scale of training set (i.e., video-only WebVid10M) with some randomly collected text-free videos and are encouraged to observe the performance improvement (FID from 9.67 to 8.19 and FVD from 484 to 441), demonstrating the scalability of Copyright © 2023, The Authors. All rights reserved.

关键词：

A2J-Transformer: Anchor-to-Joint Transformer Network for 3D Interacting Hand Pose Estimation from a Single RGB image

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Jiang, Changlong Xiao, Yang Wu, Cunlin Zhang, Mingyang Zheng, Jinghong Cao, Zhiguo Zhou, Joey Tianyi Key Laboratory of Image Processing and Intelligent Control Ministry of Education School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Alibaba Group China Singapore Singapore

3D interacting hand pose estimation from a single RGB image is a challenging task, due to serious self-occlusion and inter-occlusion towards hands, confusing similar appearance patterns between 2 hands, ill-posed joint position mapping from 2D to 3D, etc.. To address these, we propose to extend A2J-the state-of-the-art depth-based 3D single hand pose estimation method-to RGB domain under interacting hand condition. Our key idea is to equip A2J with strong local-global aware ability to well capture interacting hands’ local fine details and global articulated clues among joints jointly. To this end, A2J is evolved under Transformer’s non-local encoding-decoding framework to build A2J-Transformer. It holds 3 main advantages over A2J. First, self-attention across local anchor points is built to make them global spatial context aware to better capture joints’ articulation clues for resisting occlusion. Secondly, each anchor point is regarded as learnable query with adaptive feature learning for facilitating pattern fitting capacity, instead of having the same local representation with the others. Last but not least, anchor point locates in 3D space instead of 2D as in A2J, to leverage 3D pose prediction. Experiments on challenging InterHand 2.6M demonstrate that, A2J-Transformer can achieve state-of-the-art model-free performance (3.38mm MPJPE advancement in 2-hand case) and can also be applied to depth domain with strong generalization. The code is avaliable at https://***/ChanglongJiangGit/A2J-Transformer. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Csp-Net: Common Spatial Pattern Empowered Neural Networks for Eeg-Based Motor imagery Classification

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Jiang, Xue Meng, Lubin Chen, Xinru Xu, Yifan Wu, Dongrui Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Shenzhen Huazhong University of Science and Technology Research Institute Shenzhen518063 China

Electroencephalogram-based motor imagery (MI) classification is an important paradigm of non-invasive brain-computer interfaces. Common spatial pattern (CSP), which exploits different energy distributions on the scalp while performing different MI tasks, is very popular in MI classification. Convolutional neural networks (CNNs) have also achieved great success, due to their powerful learning capabilities. This paper proposes two CSP-empowered neural networks (CSP-Nets), which integrate knowledge-driven CSP filters with data-driven CNNs to enhance the performance in MI classification. CSP-Net-1 directly adds a CSP layer before a CNN to improve the input discriminability. CSP-Net-2 replaces a convolutional layer in CNN with a CSP layer. The CSP layer parameters in both CSP-Nets are initialized with CSP filters designed from the training data. During training, they can either be kept fixed or optimized using gradient descent. Experiments on four public MI datasets demonstrated that the two CSP-Nets consistently improved over their CNN backbones, in both within-subject and cross-subject classifications. They are particularly useful when the number of training samples is very small. Our work demonstrates the advantage of integrating knowledge-driven traditional machine learning with data-driven deep learning in EEG-based brain-computer interfaces. © 2024, The Authors. All rights reserved.

关键词： Electroencephalography

Multiple granularity network and dynamic label for domain adaptive person re-identification 4

学校读者我要写书评

暂无评论

Multiple granularity network and dynamic label for domain ad...

2021 4th International Conference on Computer Information Science and Application Technology, CISAT 2021

作者： Wang, Xile Zhang, Sihan Song, Junyu Zhang, Miaohui Henan Key Laboratory of Big Data Analysis and Processing Henan University Henan Kaifeng475004 China School of Artificial Intelligence Henan University Henan Kaifeng475004 China

The domain adaptive person re-identification (Re-ID) has be more popular among researchers. Because it can save a lot of resources as it only exploits the source domain knowledge and does not need the complex annotation efforts in target domain. It aims to extend a model trained on a labeled dataset to another dataset which is unlabeled. Many works reduce feature distribution gap between two different datasets to solve the problem. However, these works ignore the problem which is the variations within an unlabeled dataset. In the paper, we propose a domain adaptive person Re-ID framework based on multiple granularity network and dynamic label (MGDL). Specifically, we send the images of two different datasets into multiple granularity network at the same time for joint training to reduce feature distribution gap which is between the two different datasets. The network is trained by two different kinds of pseudo labels, namely, conservative label and radical label. The two kinds of pseudo labels are used to alternating pull and push the feature distribution in the target domain to reduce the variations within an unlabeled dataset. Experiments have shown that the MGDL achieves competitive performance in person Re-ID which is under the cross-domain setting. © Content from this work may be used under the terms of the Creative Commons Attribution 3.0 Licence.

关键词：

Depth Guided Cross-modal Residual Adaptive Network for RGB-D Salient Object Detection 2

学校读者我要写书评

暂无评论

Depth Guided Cross-modal Residual Adaptive Network for RGB-D...

2021 2nd International Workshop on Electronic communication and Artificial intelligence, IWECAI 2021

作者： Zhao, Zhengyun Yang, Qingpeng Yang, Shangqin Wang, Jun School of Artificial Intelligence Henan University Kaifeng Henan475004 China Henan Key Laboratory of Big Data Analysis and Processing Henan University Kaifeng Henan475004 China

Depth modal features can provide complementary information for salient object detection (SOD). Most of the existing RGB-D SOD methods focus on fully combining RGB and Depth modal features without distinguishing them. In this paper, we propose a new depth guided cross-modal residual adaptive network for RGB-D SOD. We use two independent resnet-50 to extract the features of the two modes respectively. Then the cross-modal channel-wise refinement module is designed to obtain complementary modal information. We design a crossmodal guided module to make complementary modal information guide RGB image feature extraction. Finally, the residual adaptive selection module is used to enhance the spatial mutual concerns between the two modal features to achieve multimodal information fusion. Experimental results show that our method can achieve a more reasonable fusion state of RGB and Depth, and verify the effectiveness of our final saliency model. © Published under licence by IOP Publishing Ltd.

关键词： Object detection

Hybrid Data-Free Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tang, Jialiang Chen, Shuo Gong, Chen School of Computer Science and Engineering Nanjing University of Science and Technology China Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education China Jiangsu Key Laboratory of Image and Video Understanding for Social Security China Center for Advanced Intelligence Project RIKEN Japan Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

Data-free knowledge distillation aims to learn a compact student network from a pre-trained large teacher network without using the original training data of the teacher network. Existing collection-based and generation-based methods train student networks by collecting massive real examples and generating synthetic examples, respectively. However, they inevitably become weak in practical scenarios due to the difficulties in gathering or emulating sufficient real-world data. To solve this problem, we propose a novel method called Hybrid Data-Free Distillation (HiDFD), which leverages only a small amount of collected data as well as generates sufficient examples for training student networks. Our HiDFD comprises two primary modules, i.e., the teacher-guided generation and student distillation. The teacher-guided generation module guides a Generative Adversarial Network (GAN) by the teacher network to produce high-quality synthetic examples from very few real-world collected examples. Specifically, we design a feature integration mechanism to prevent the GAN from overfitting and facilitate the reliable representation learning from the teacher network. Meanwhile, we drive a category frequency smoothing technique via the teacher network to balance the generative training of each category. In the student distillation module, we explore a data inflation strategy to properly utilize a blend of real and synthetic data to train the student network via a classifier-sharing-based feature alignment technique. Intensive experiments across multiple benchmarks demonstrate that our HiDFD can achieve state-of-the-art performance using 120 times less collected data than existing methods. Code is available at https://***/tangjialiang97/HiDFD. Copyright © 2024, The Authors. All rights reserved.

关键词： Students

Active Compliance Variable Impedance Control of Lower Supernumerary Robotic Limb*

学校读者我要写书评

暂无评论

Active Compliance Variable Impedance Control of Lower Supern...

International Conference on Advanced Robotics and Mechatronics (ICARM)

作者： Jun Huo Xinyi Wang Bo Yang Qingyang Yan Zhaofan Yuan Jian Huang Hubei Key Laboratory of Brain-inspired Intelligent Systems and the Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan China China Telecom Research Institute Guangdong China China Ship Development and Design Centre Wuhan China

ISBN: (数字)9798350385724

ISBN: (纸本)9798350385731

In human-robot systems, ensuring user safety amidst external disturbances remains paramount. An efficient variable impedance control (VIC) technique is designed to enhance human-robot interaction, particularly in scenarios with external force disturbances. By dynamically adapting impedance parameters in response to detected changes in contact force, VIC achieves heightened system flexibility. The robust stability of the VIC system is rigorously established using the Lyapunov approach, with a parameter update law tailored for a lower limb supernumerary robotic limb robot. Through simulations and experiments, the efficacy of the system in maintaining a specific safety margin is demonstrated. Moreover, the technique offers a pragmatic solution to accommodate individual variations in walking gait and cross-user disparities in interaction. The proposed method contributes to advancing the safety and adaptability of cross-user human-robot systems.

关键词： Legged locomotion Adaptation models Mechatronics Robust stability Force Human-robot interaction Safety Impedance Robots Pragmatics

Network Embedding with Topology-Aware Textual Representations 18th

学校读者我要写书评

暂无评论

Network Embedding with Topology-Aware Textual Representation...

18th Pacific Rim International Conference on Artificial intelligence, PRICAI 2021

作者： Chen, Jiaxing Xu, Zenan Su, Qinliang School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education Guangzhou China

ISBN: (纸本)9783030891879

Textual network embedding aims to learn meaningful low-dimensional representations for vertices with the consideration of the associated texts. When learning the representations for texts in network embedding, existing methods mostly only exploit information from neighboring texts (i.e., contexts), while rarely taking advantages of the valuable network topological (structural) information. To bridge the gap, in this paper, a model based on adaptive-filter convolutional neural networks (CNN) is developed, in which the filters are adapted to local network topologies, rather than clamped to fixed values as in traditional CNNs. The dependency enables the learned text representations to be aware of local network topologies. It is shown that the proposed topology-aware representations can be viewed as a complement to existing context-aware ones. When the two are used together, experimental results on three real-world benchmarks demonstrate that significant performance improvements on the tasks of link prediction and vertex classification. © 2021, Springer Nature Switzerland AG.

关键词： Network topology

MRCN: A Novel Modality Restitution and Compensation Network for Visible-Infrared Person Re-identification

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Yukang Yan, Yan Li, Jie Wang, Hanzi Fujian Key Laboratory of Sensing and Computing for Smart City School of Informatics Xiamen University China Video and Image Processing System Laboratory School of Electronic Engineering Xidian University Xi’an China Shanghai Artificial Intelligence Laboratory Shanghai China

Visible-infrared person re-identification (VI-ReID), which aims to search identities across different spectra, is a challenging task due to large cross-modality discrepancy between visible and infrared images. The key to reduce the discrepancy is to filter out identity-irrelevant interference and effectively learn modality-invariant person representations. In this paper, we propose a novel Modality Restitution and Compensation Network (MRCN) to narrow the gap between the two modalities. Specifically, we first reduce the modality discrepancy by using two Instance Normalization (IN) layers. Next, to reduce the influence of IN layers on removing discriminative information and to reduce modality differences, we propose a Modality Restitution Module (MRM) and a Modality Compensation Module (MCM) to respectively distill modality-irrelevant and modality-relevant features from the removed information. Then, the modality-irrelevant features are used to restitute to the normalized visible and infrared features, while the modality-relevant features are used to compensate for the features of the other modality. Furthermore, to better disentangle the modality-relevant features and the modality-irrelevant features, we propose a novel Center-Quadruplet Causal (CQC) loss to encourage the network to effectively learn the modality-relevant features and the modality-irrelevant features. Extensive experiments are conducted to validate the superiority of our method on the challenging SYSU-MM01 and RegDB datasets. More remarkably, our method achieves 95.1% in terms of Rank-1 and 89.2% in terms of mAP on the RegDB dataset. Copyright © 2023, The Authors. All rights reserved.

关键词： Infrared imaging