检索结果-内蒙古大学图书馆

End-to-End Quantum Vision Transformer: Towards Practical Quantum Speedup in Large-Scale Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xue, Cheng Chen, Zhao-Yun Zhuang, Xi-Ning Wang, Yun-Jie Sun, Tai-Ping Wang, Jun-Chao Liu, Huan-Yu Wu, Yu-Chun Wang, Zi-Lei Guo, Guo-Ping Institute of Artificial Intelligence Hefei Comprehensive National Science Center Anhui Hefei230088 China CAS Key Laboratory of Quantum Information University of Science and Technology of China Anhui Hefei230026 China CAS Center For Excellence in Quantum Information and Quantum Physics University of Science and Technology of China Anhui Hefei230026 China Origin Quantum Computing Company Limited Anhui Hefei230088 China Institute of Advanced Technology University of Science and Technology of China Anhui Hefei230031 China Laboratory for Advanced Computing and Intelligence Engineering Zhengzhou450001 China National Engineering Laboratory for Brain-inspired Intelligence Technology and Application University of Science and Technology of China Hefei230027 China

The field of quantum deep learning presents significant opportunities for advancing computational capabilities, yet it faces a major obstacle in the form of the "information loss problem" due to the inherent limitations of the necessary quantum tomography in scaling quantum deep neural networks. This paper introduces an end-to-end Quantum Vision Transformer (QViT), which incorporates an innovative quantum residual connection technique, to overcome these challenges and therefore optimize quantum computing processes in deep learning. Our thorough complexity analysis of the QViT reveals a theoretically exponential and empirically polynomial speedup, showcasing the model’s efficiency and potential in quantum computing applications. We conducted extensive numerical tests on modern, large-scale transformers and datasets, establishing the QViT as a pioneering advancement in applying quantum deep neural networks in practical scenarios. Our work provides a comprehensive quantum deep learning paradigm, which not only demonstrates the versatility of current quantum linear algebra algorithms but also promises to enhance future research and development in quantum deep learning1 Copyright © 2024, The Authors. All rights reserved.

关键词： Deep neural networks

View-decoupled Transformer for Person Re-identification under Aerial-ground Camera Network

学校读者我要写书评

暂无评论

View-decoupled Transformer for Person Re-identification unde...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Quan Zhang Lei Wang Vishal M. Patel Xiaohua Xie Jianhuang Lai School of Computer Science and Engineering Sun Yat-Sen University China Department of Electrical and Computer Engineering Johns Hopkins University USA Pazhou Lab (HuangPu) Guangdong China Guangdong Province Key Laboratory of Information Security Technology Guangdong China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Existing person re-identification methods have achieved remarkable advances in appearance-based identity association across homogeneous cameras, such as ground-ground matching. However, as a more practical scenario, aerial-ground person re-identification (AGPReID) among heterogeneous cameras has received minimal attention. To alleviate the disruption of discriminative identity representation by dramatic view discrepancy as the most significant challenge in AGPReID, the view-decoupled transformer (VDT) is proposed as a simple yet effective framework. Two major components are designed in VDT to decouple view-related and view-unrelated features, namely hierarchical subtractive separation and orthogonal loss, where the former separates these two features inside the VDT, and the latter constrains these two to be independent. In addition, we contribute a large-scale AGPReID dataset called CARGO, consisting of five/eight aerial/ground cameras, 5,000 identities, and 108,563 images. Experiments on two datasets show that VDT is a feasible and effective solution for AGPReID, surpassing the previous method on mAP/Rank1 by up to 5.0%/2.7% on CARGO and 3.7%/5.2% on AG-ReID, keeping the same magnitude of computational complexity. Our project is available at https://***/LinlyAC/VDT-AGPReID.

关键词： Computer vision Cameras Transformers Pattern recognition Computational complexity Identification of persons

CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cheng, Zihui Chen, Qiguang Zhang, Jin Fei, Hao Feng, Xiaocheng Che, Wanxiang Li, Min Qin, Libo School of Computer Science and Engineering Central South University China Key Laboratory of Data Intelligence and Advanced Computing in Provincial Universities Soochow University China Research Center for SCIR Harbin Institute of Technology Harbin China National University of Singapore Singapore

Large Vision-Language Models (LVLMs) have recently demonstrated amazing success in multi-modal tasks, including advancements in Multi-modal Chain-of-Thought (MCoT) reasoning. Despite these successes, current benchmarks still follow a traditional paradigm with multi-modal input and text-modal output, which leads to significant drawbacks such as missing visual operations and vague expressions. Motivated by this, we introduce a novel Chain of Multi-modal Thought (CoMT) benchmark to address these limitations. Different from the traditional MCoT benchmark, CoMT requires both multi-modal input and multi-modal reasoning output, aiming to mimic human-like reasoning that inherently integrates visual operations. Specifically, CoMT consists of four categories: (1) Visual Creation, (2) Visual Deletion, (3) Visual Update, and (4) Visual Selection to comprehensively explore complex visual operations and concise expression in real scenarios. We evaluate various LVLMs and strategies on CoMT, revealing some key insights into the capabilities and limitations of the current approaches. We hope that CoMT can inspire more breakthroughs on introducing multi-modal generation into the reasoning process. The project page is available at https://***/czhhzc/CoMT. © 2024, CC BY-NC-SA.

关键词： Chains

Plug-and-Hide: Provable and Adjustable Diffusion Generative Steganography

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhu, Jiahao Chen, Zixuan Yang, Lingxiao Xie, Xiaohua Zhou, Yi The School of Computer Science and Engineering Sun Yat-Sen University China The Guangdong Province Key Laboratory of Information Security Technology China The Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education Guangzhou510006 China The Zhongshan School of Medicine Sun Yat-Sen University China

Generative Steganography (GS) is a novel technique that utilizes generative models to conceal messages without relying on cover images. Contemporary GS algorithms leverage the powerful generative capabilities of Diffusion Models (DMs) to create high-fidelity stego images. However, these algorithms, while yielding relatively satisfactory generation outcomes and message extraction accuracy, significantly alter modifications to the initial Gaussian noise of DMs, thereby compromising steganographic security. In this paper, we rethink the trade-off among image quality, steganographic security, and message extraction accuracy within Diffusion Generative Steganography (DGS) settings. Our findings reveal that the normality of initial noise of DMs is crucial to these factors and can offer theoretically grounded guidance for DGS design. Based on this insight, we propose a Provable and Adjustable Message Mapping (PA-B2G) approach. It can, on one hand, theoretically guarantee reversible encoding of bit messages from arbitrary distributions into standard Gaussian noise for DMs. On the other hand, its adjustability provides a more natural and fine-grained way to trade off image quality, steganographic security, and message extraction accuracy. By integrating PAB2G with a probability flow ordinary differential equation, we establish an invertible mapping between secret messages and stego images. PA-B2G can be seamlessly incorporated with most mainstream DMs, such as the Stable Diffusion, without necessitating additional training or fine-tuning. Comprehensive experiments corroborate our theoretical insights regarding the trade-off in DGS settings and demonstrate the effectiveness of our DGS algorithm in producing high-quality stego images while preserving desired levels of steganographic security and extraction accuracy. © 2024, CC BY-NC-ND.

关键词： Gaussian noise (electronic)

The Enemy of My Enemy is My Friend: Exploring Inverse Adversaries for Improving Adversarial Training

学校读者我要写书评

暂无评论

The Enemy of My Enemy is My Friend: Exploring Inverse Advers...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Junhao Dong Seyed-Mohsen Moosavi-Dezfooli Jianhuang Lai Xiaohua Xie School of Computer Science and Engineering Sun Yat-Sen University China Imperial College London UK Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Although current deep learning techniques have yielded superior performance on various computer vision tasks, yet they are still vulnerable to adversarial examples. Adversarial training and its variants have been shown to be the most effective approaches to defend against adversarial examples. A particular class of these methods regularize the difference between output probabilities for an adversarial and its corresponding natural example. However, it may have a negative impact if a natural example is misclassified. To circumvent this issue, we propose a novel adversarial training scheme that encourages the model to produce similar output probabilities for an adversarial example and its “inverse adversarial” counterpart. Particularly, the counterpart is generated by maximizing the likelihood in the neighborhood of the natural example. Extensive experiments on various vision datasets and architectures demonstrate that our training method achieves state-of-the-art robustness as well as natural accuracy among robust models. Furthermore, using a universal version of inverse adversarial examples, we improve the performance of single-step adversarial training techniques at a low computational cost.

关键词：

Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fu, Shenghao Yan, Junkai Yang, Qize Wei, Xihan Xie, Xiaohua Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Peng Cheng Laboratory Shenzhen518055 China Tongyi Lab Alibaba Group China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Guangdong Province Key Laboratory of Information Security Technology China Guangdong Guangzhou510555 China

Recent vision foundation models can extract universal representations and show impressive abilities in various tasks. However, their application on object detection is largely overlooked, especially without fine-tuning them. In this work, we show that frozen foundation models can be a versatile feature enhancer, even though they are not pre-trained for object detection. Specifically, we explore directly transferring the high-level image understanding of foundation models to detectors in the following two ways. First, the class token in foundation models provides an in-depth understanding of the complex scene, which facilitates decoding object queries in the detector’s decoder by providing a compact context. Additionally, the patch tokens in foundation models can enrich the features in the detector’s encoder by providing semantic details. Utilizing frozen foundation models as plug-and-play modules rather than the commonly used backbone can significantly enhance the detector’s performance while preventing the problems caused by the architecture discrepancy between the detector’s backbone and the foundation model. With such a novel paradigm, we boost the SOTA query-based detector DINO from 49.0% AP to 51.9% AP (+2.9% AP) and further to 53.8% AP (+4.8% AP) by integrating one or two foundation models respectively, on the COCO validation set after training for 12 epochs with R50 as the detector’s backbone. Copyright © 2024, The Authors. All rights reserved.

关键词： Decoding

Temporal Information Reconstruction and Non-Aligned Residual in Spiking Neural Networks for Speech Classification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Qi Wang, Huamin Shen, Hangchi Duan, Shukai Wen, Shiping Huang, Tingwen College of Artificial Intelligence Southwest University Chongqing400715 China Chongqing Key Laboratory of Brain Inspired Computing and Intelligent Chips Chongqing400715 China College of Information Engineering University of Technology Sydney Australian Institute of Artificial Intelligence Sydney2007 Australia Faculty of Computer Science and Control Engineering Shenzhen University of Advanced Technology Shenzhen518055 China

Recently, it can be noticed that most models based on spiking neural networks (SNNs) only use a same level temporal resolution to deal with speech classification problems, which makes these models cannot learn the information of input data at different temporal scales. Additionally, owing to the different time lengths of the data before and after the sub-modules of many models, the effective residual connections cannot be applied to optimize the training processes of these models. To solve these problems, on the one hand, we reconstruct the temporal dimension of the audio spectrum to propose a novel method named as Temporal Reconstruction (TR) by referring the hierarchical processing process of the human brain for understanding speech. Then, the reconstructed SNN model with TR can learn the information of input data at different temporal scales and model more comprehensive semantic information from audio data because it enables the networks to learn the information of input data at different temporal resolutions. On the other hand, we propose the Non-Aligned Residual (NAR) method by analyzing the audio data, which allows the residual connection can be used in two audio data with different time lengths. We have conducted plentiful experiments on the Spiking Speech Commands (SSC), the Spiking Heidelberg Digits (SHD), and the Google Speech Commands v0.02 (GSC) datasets. According to the experiment results, we have achieved the state-of-the-art (SOTA) result 81.02% on SSC for the test classification accuracy of all SNN models, and we have obtained the SOTA result 96.04% on SHD for the classification accuracy of all models. Furthermore, on the non-spiking dataset GSC, we can achieve a test classification accuracy of 95.63%, which surpasses the original baseline SNN-Delays. Meanwhile, the method proposed in this paper can speed up the model’s processing speed and reduce its energy consumption. Copyright © 2024, The Authors. All rights reserved.

关键词： Spatio-temporal data

Towards Securing ASCON Implementation by Inner Product Masking

学校读者我要写书评

暂无评论

Towards Securing ASCON Implementation by Inner Product Maski...

IEEE International Conference on Trust, Security and Privacy in computing and Communications (TrustCom)

作者： Yuming Liu Wei Cheng Jihao Fan Yongbin Zhou School of Cyber Science and Engineering Nanjing University of Science and Technology Nanjing China LTCI Télécom Paris IP Paris and Secure-IC S.A.S Paris France Laboratory for Advanced Computing and Intelligence Engineering Wuxi China Institute of Information Engineering Chinese Academy of Sciences Beijing China

ISBN: (数字)9798331506209

ISBN: (纸本)9798331506216

Ascon algorithm has been selected by NIST for standardization of the lightweight cryptography, that will be used in embedded systems, IoT devices, and other resource-constrained devices. As a lightweight cryptographic algorithm, Ascon implementation requires less computation and memory. However, the current implementations of Ascon algorithms and existing protection schemes are vulnerable to side-channel attacks. In this paper, we propose an IPM (inner product masking) based protection for Ascon implementations. Our new masking scheme can prevent side-channel attacks from successfully retrieving the key, even when the leakage levels are nearly identical. Alternatively, the cost of obtaining the key can increase by several orders of magnitude, making the attack significantly more challenging. The proposed scheme is experimentally validated by porting it into a STM32F407 microcontroller and conducting correlation power analysis (CPA) and template attack (TA) on the collected power traces. The experimental results show that while CPA can successfully retrieve the keys from both the unprotected Ascon implementation and the Boolean-masked Ascon implementation, they fail against the new masking scheme. Although TA attacks can break the new scheme, they require approximately 50 times more data compared to attacking the Boolean mask. This demonstrates that the new masking scheme offers significantly greater security than both the Boolean mask and the unprotected implementations.

关键词： Resistance Privacy Microcontrollers Refining Memory management Side-channel attacks Standardization NIST Internet of Things Protection

Improving Visual Quality of Image Synthesis by A Token-based Generator with Transformers 35

学校读者我要写书评

暂无评论

Improving Visual Quality of Image Synthesis by A Token-based...

35th Conference on Neural Information Processing Systems, NeurIPS 2021

作者： Zeng, Yanhong Yang, Huan Wang, Jianbo Fu, Jianlong School of Computer Science and Engineering Sun Yat-sen University China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Microsoft Research Asia China University of Tokyo Japan

ISBN: (纸本)9781713845393

We present a new perspective of achieving image synthesis by viewing this task as a visual token generation problem. Different from existing paradigms that directly synthesize a full image from a single input (e.g., a latent code), the new formulation enables a flexible local manipulation for different image regions, which makes it possible to learn content-aware and fine-grained style control for image synthesis. Specifically, it takes as input a sequence of latent tokens to predict the visual tokens for synthesizing an image. Under this perspective, we propose a token-based generator (i.e.,TokenGAN). Particularly, the TokenGAN inputs two semantically different visual tokens, i.e., the learned constant content tokens and the style tokens from the latent space. Given a sequence of style tokens, the TokenGAN is able to control the image synthesis by assigning the styles to the content tokens by attention mechanism with a Transformer. We conduct extensive experiments and show that the proposed TokenGAN has achieved state-of-the-art results on several widelyused image synthesis benchmarks, including FFHQ and LSUN CHURCH with different resolutions. In particular, the generator is able to synthesize high-fidelity images with 1024 × 1024 size, dispensing with convolutions entirely. © 2021 Neural information processing systems foundation. All rights reserved.

关键词： Image enhancement