检索结果-内蒙古大学图书馆

Fully Learnable Group Convolution for Acceleration of Deep Neural Networks

学校读者我要写书评

暂无评论

Fully Learnable Group Convolution for Acceleration of Deep N...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Xijun Wang Meina Kan Shiguang Shan Xilin Chen Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China CAS Center for Excellence in Brain Science and Intelligence Technology Shanghai China

ISBN: (纸本)9781728132945

Benefitted from its great success on many tasks, deep learning is increasingly used on low-computational-cost devices, e.g. smartphone, embedded devices, etc. To reduce the high computational and memory cost, in this work, we propose a fully learnable group convolution module (FLGC for short) which is quite efficient and can be embedded into any deep neural networks for acceleration. Specifically, our proposed method automatically learns the group structure in the training stage in a fully end-to-end manner, leading to a better structure than the existing pre-defined, two-steps, or iterative strategies. Moreover, our method can be further combined with depthwise separable convolution, resulting in 5 times acceleration than the vanilla Resnet50 on single CPU. An additional advantage is that in our FLGC the number of groups can be set as any value, but not necessarily 2^k as in most existing methods, meaning better tradeoff between accuracy and speed. As evaluated in our experiments, our method achieves better performance than existing learnable group convolution and standard group convolution when using the same number of groups.

关键词： acceleration Convolution numerical invariant Group Structure Neural network

DOA Estimation for RIS Assisted Multi-User Wireless Communication System with Squared-Sine Error Criterion and Lawson Norm Constraint

学校读者我要写书评

暂无评论

DOA Estimation for RIS Assisted Multi-User Wireless Communic...

IEEE International Conference on Electronic Information and Communication Technology (ICEICT)

作者： Canping Yu Li Li Yingsong Li Dazhi Sun Liping Li Zhixiang Huang Tian Hong Loh Key Laboratory of Intelligent Computing and Signal Processing Ministry of Education Anhui University China Aircraft Airworthiness Institute China Academy of Civil Aviation Science and Technology (Center of Aviation Safety Technology CAST) Beijing China Electromagnetic and Eletrochemical Technologies Department National Physical Laboratory Teddington Middlesex UK

Reconfigurable intelligent surface (RIS) can control the phase of reflected signals, and has become a hotspot in the field of wireless communication. In this paper, we propose a direction-of-arrival (DOA) estimation algorithm for RIS assisted multi-user wireless communication systems with squared-sine error criterion and Lawson norm constraint. This algorithm utilizes the sparsity of the signal and can achieve fast and accurate DOA estimation under the interference of impulsive noises. To suppress the line-of-sight (LOS) interference and thus to maximize the received signal-to-interference-plus-noise ratio (SINR), we also propose a random optimal phase selection (ROPS) technique to design the control matrix of RIS elements. The simulation results show that the proposed algorithm performs better than the existing methods.

关键词：

Design of Ultra-Wideband Terahertz Wavelength Division Multiplexer Based on Inverse Design Algorithm

学校读者我要写书评

暂无评论

Design of Ultra-Wideband Terahertz Wavelength Division Multi...

International Applied Computational Electromagnetics Society Symposium (ACES)

作者： Zhicheng Xiao Kaihong Song Ming Fang Anhui Province Key Laboratory of Target Recognition and Feature Extraction Anhui University Anhui China Information Materials and Intelligent Sensing Laboratory of Anhui Province Anhui University Hefei Anhui China Anhui Province Key Laboratory of Target Recognition and Feature Extraction Luan Anhui China Key Laboratory of Intelligent Computing and Signal Processing Ministry of Education Anhui University Hefei China State Key Laboratory of Millimeter Waves Southeast University Nanjing China

Integrated photonic gadgets play a key function in lots of fields. The process of designing silicon optical devices mostly relies on experience to manually adjust the subtle parameters, which is laborious and ineffective. The proposed inverse design algorithms can simplify the design process of silicon optical devices, improve the design efficiency, and realize multifunctional compact devices. In this paper, two inverse design based methods are proposed to design terahertz band wavelength division multiplexer(WDM). One is based on topology optimization to design a 4-channel ultra-wideband WDM and the other employs a direct binary search technique to create a 3-channel ultra-wideband terahertz WDM, both of which have high transmission, low interference and high integration. Multifunctional photonics devices are predicted to be increasingly used in modern photonic integrated circuits due to these advantages.

关键词： Performance evaluation Photonic integrated circuits Wavelength division multiplexing Prediction algorithms Silicon Topology Optical devices

Learning Music-Dance Representations through Explicit-Implicit Rhythm Synchronization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yu, Jiashuo Pu, Junfu Cheng, Ying Feng, Rui Shan, Ying The School of Computer Science Shanghai Key Lab of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University Shanghai200438 China The Academy for Engineering and Technology Fudan University Shanghai200438 China The Applied Research Center PCG Tencent Shenzhen518000 China

Although audio-visual representation has been proven to be applicable in many downstream tasks, the representation of dancing videos, which is more specific and always accompanied by music with complex auditory contents, remains challenging and uninvestigated. Considering the intrinsic alignment between the cadent movement of the dancer and music rhythm, we introduce MuDaR, a novel Music-Dance Representation learning framework to perform the synchronization of music and dance rhythms both in explicit and implicit ways. Specifically, we derive the dance rhythms based on visual appearance and motion cues inspired by the music rhythm analysis. Then the visual rhythms are temporally aligned with the music counterparts, which are extracted by the amplitude of sound intensity. Meanwhile, we exploit the implicit coherence of rhythms implied in audio and visual streams by contrastive learning. The model learns the joint embedding by predicting the temporal consistency between audio-visual pairs. The music-dance representation, together with the capability of detecting audio and visual rhythms, can further be applied to three downstream tasks: (a) dance classification, (b) music-dance retrieval, and (c) music-dance retargeting. Extensive experiments demonstrate that our proposed framework outperforms other self-supervised methods by a large margin. © 2022, CC BY.

关键词： Supervised learning

Spatiotemporal Activity Modeling via Hierarchical Cross-Modal Embedding : Extended Abstract

学校读者我要写书评

暂无评论

Spatiotemporal Activity Modeling via Hierarchical Cross-Moda...

International Conference on Data Engineering

作者： Yang Liu Xiang Ao Linfeng Dong Chao Zhang Jin Wang Qing He Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China Zhejiang University Hangzhou China College of Computing Georgia Tech United States Megagon Labs Mountain View United States

With the ever-increasing urbanization process, modeling people’s spatiotemporal activities from their online traces has become a crucial task. State-of-the-art methods for this task rely on cross-modal embedding, which maps items from different modalities (e.g., location, time, text) into the same latent space. Despite their inspiring results, existing cross-modal embedding methods merely capture co-occurrences between items without modeling their high-order interactions. In this paper, we first construct the user interaction graph and the activity graph from raw data records and propose a hierarchical cross-modal embedding method that takes the high-order relationships into consideration. We introduce both inter-record and intra-record meta-graph structures, which enable learning distributed representations that preserve high-order proximities across graphs from different layers. Our empirical experiments on three real-world datasets demonstrate that our method not only outperforms state-of-the-art methods for spatiotemporal activity prediction but also captures cross-modal proximity at a finer granularity.

关键词：

Let's play music: Audio-driven performance video generation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhu, Hao Li, Yi Zhu, Feixia Zheng, Aihua He, Ran Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education School of Computer Science and Technology Anhui University Hefei China CASIA Beijing China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China Center for Excellence in Brain Science and Intelligence Technology CAS Beijing China

We propose a new task named Audio-driven Performance Video Generation (APVG), which aims to synthesize the video of a person playing a certain instrument guided by a given music audio clip. It is a challenging task to generate the high-dimensional temporal consistent videos from low-dimensional audio modality. In this paper, we propose a multistaged framework to achieve this new task to generate realistic and synchronized performance video from given music. Firstly, we provide both global appearance and local spatial information by generating the coarse videos and keypoints of body and hands from a given music respectively. Then, we propose to transform the generated keypoints to heatmap via a differentiable space transformer, since the heatmap offers more spatial information but is harder to generate directly from audio. Finally, we propose a Structured Temporal UNet (STU) to extract both intra-frame structured information and inter-frame temporal consistency. They are obtained via graph-based structure module, and CNN-GRU based high-level temporal module respectively for final video generation. Comprehensive experiments validate the effectiveness of our proposed framework. Copyright © 2020, The Authors. All rights reserved.

关键词： Music

Transformer RGBT Tracking with Spatio-Temporal Multimodal Tokens

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Dengdi Pan, Yajie Lu, Andong Li, Chenglong Luo, Bin The Key Laboratory of Intelligent Computing & Signal Processing Ministry of Education School of Artificial Intelligence Anhui University Hefei230601 China Institute of Artificial Intelligence Hefei Comprehensive National Science Center Hefei230026 China The Anhui Provincial Key Laboratory of Multimodal Cognitive Computation School of Computer Science and Technology Anhui University Hefei230601 China

Many RGBT tracking researches primarily focus on modal fusion design, while overlooking the effective handling of target appearance changes. While some approaches have introduced historical frames or fuse and replace initial templates to incorporate temporal information, they have the risk of disrupting the original target appearance and accumulating errors over time. To alleviate these limitations, we propose a novel Transformer RGBT tracking approach, which mixes spatio-temporal multimodal tokens from the static multimodal templates and multimodal search regions in Transformer to handle target appearance changes, for robust RGBT tracking. We introduce independent dynamic template tokens to interact with the search region, embedding temporal information to address appearance changes, while also retaining the involvement of the initial static template tokens in the joint feature extraction process to ensure the preservation of the original reliable target appearance information that prevent deviations from the target appearance caused by traditional temporal updates. We also use attention mechanisms to enhance the target features of multimodal template tokens by incorporating supplementary modal cues, and make the multimodal search region tokens interact with multimodal dynamic template tokens via attention mechanisms, which facilitates the conveyance of multimodal-enhanced target change information. Our module is inserted into the transformer backbone network and inherits joint feature extraction, search-template matching, and cross-modal interaction. Extensive experiments on three RGBT benchmark datasets show that the proposed approach maintains competitive performance compared to other state-of-the-art tracking algorithms while running at 39.1 FPS. Copyright © 2024, The Authors. All rights reserved.

关键词： Feature extraction

LightXML: Transformer with dynamic negative sampling for high-performance extreme multi-label text classification

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Jiang, Ting Wang, Deqing Sun, Leilei Yang, Huayi Zhao, Zhengyang Zhuang, Fuzhen SKLSDE and BDBC Lab Beihang University Beijing China Key Lab of Intelligent Information Processing of CAS Institute of Computing Technology CAS Beijing China Beijing Advanced Innovation Center for Imaging Theory and Technology Academy for Multidisciplinary Studies Capital Normal University Beijing China

Extreme Multi-label text Classification (XMC) is a task of finding the most relevant labels from a large label set. Nowadays deep learning-based methods have shown significant success in XMC. However, the existing methods (e.g., AttentionXML and X-Transformer etc) still suffer from 1) combining several models to train and predict for one dataset, and 2) sampling negative labels statically during the process of training label ranking model, which reduces both the efficiency and accuracy of the model. To address the above problems, we proposed LightXML, which adopts end-to-end training and dynamic negative labels sampling. In LightXML, we use generative cooperative networks to recall and rank labels, in which label recalling part generates negative and positive labels, and label ranking part distinguishes positive labels from these labels. Through these networks, negative labels are sampled dynamically during label ranking part training by feeding with the same text representation. Extensive experiments show that LightXML outperforms state-of-the-art methods in five extreme multi-label datasets with much smaller model size and lower computational complexity. In particular, on the Amazon dataset with 670K labels, LightXML can reduce the model size up to 72% compared to AttentionXML. Our code is available at http://***/kongds/LightXML. Copyright © 2021, The Authors. All rights reserved.

关键词： Deep learning

Cross-Modal Object Tracking via Modality-Aware Fusion Network and A Large-Scale Dataset

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Lei Zhang, Mengya Li, Cheng Li, Chenglong Tang, Jin The Information Materials and Intelligent Sensing Laboratory of Anhui Province Anhui Provincial Key Laboratory of Multimodal Cognitive Computation School of Artificial Intelligence Anhui University Hefei230601 China The Information Materials and Intelligent Sensing Laboratory of Anhui Province Key Laboratory of Intelligent Computing and Signal Processing The Ministry of Education Anhui Provincial Key Laboratory of Multimodal Cognitive Computation School of Computer Science and Technology Anhui University Hefei230601 China The School of Computer Science and Technology Anhui University Hefei230601 China

Visual tracking often faces challenges such as invalid targets and decreased performance in low-light conditions when relying solely on RGB image sequences. While incorporating additional modalities like depth and infrared data has proven effective, existing multi-modal imaging platforms are complex and lack real-world applicability. In contrast, near-infrared (NIR) imaging, commonly used in surveillance cameras, can switch between RGB and NIR based on light intensity. However, tracking objects across these heterogeneous modalities poses significant challenges, particularly due to the absence of modality switch signals during tracking. To address these challenges, we propose an adaptive cross-modal object tracking algorithm called Modality-Aware Fusion Network (MAFNet). MAFNet efficiently integrates information from both RGB and NIR modalities using an adaptive weighting mechanism, effectively bridging the appearance gap and enabling a modality-aware target representation. It consists of two key components: an adaptive weighting module and a modality-specific representation module. The adaptive weighting module predicts fusion weights to dynamically adjust the contribution of each modality, while the modality-specific representation module captures discriminative features specific to RGB and NIR modalities. MAFNet offers great flexibility as it can effortlessly integrate into diverse tracking frameworks. With its simplicity, effectiveness, and efficiency, MAFNet outperforms state-of-the-art methods in cross-modal object tracking. To validate the effectiveness of our algorithm and overcome the scarcity of data in this field, we introduce CMOTB, a comprehensive and extensive benchmark dataset for cross-modal object tracking. CMOTB consists of 61 categories and 1000 video sequences, comprising a total of over 799K frames. We believe that our proposed method and dataset offer a strong foundation for advancing cross-modal object tracking research. The dataset, toolkit, a

关键词： Large dataset