检索结果-内蒙古大学图书馆

AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Zhuguanyu Chen, Jiaxin Zhong, Hanwen Huang, Di Wang, Yunhong State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China School of Computer Science and Engineering Beihang University Beijing China

Vision Transformer (ViT) has become one of the most prevailing fundamental backbone networks in the computer vision community. Despite the high accuracy, deploying it in real applications raises critical challenges including the high computational cost and inference latency. Recently, the post-training quantization (PTQ) technique has emerged as a promising way to enhance ViT's efficiency. Nevertheless, existing PTQ approaches for ViT suffer from the inflexible quantization on the post-Softmax and post-GELU activations that obey the power-law-like distributions. To address these issues, we propose a novel non-uniform quantizer, dubbed the Adaptive Logarithm AdaLog (AdaLog) quantizer. It optimizes the logarithmic base to accommodate the power-law-like distribution of activations, while simultaneously allowing for hardware-friendly quantization and de-quantization. By employing the bias reparameterization, the AdaLog quantizer is applicable to both the post-Softmax and post-GELU activations. Moreover, we develop an efficient Fast Progressive Combining Search (FPCS) strategy to determine the optimal logarithm base for AdaLog, as well as the scaling factors and zero points for the uniform quantizers. Extensive experimental results on public benchmarks demonstrate the effectiveness of our approach for various ViT-based architectures and vision tasks including classification, object detection, and instance segmentation. Code is available at https://***/GoatWu/AdaLog. Copyright © 2024, The Authors. All rights reserved.

关键词： Chemical activation

PGNeXt: High-Resolution Salient Object Detection via Pyramid Grafting Network

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xia, Changqun Xie, Chenxi He, Zhentao Yu, Tianshu Li, Jia The Peng Cheng Laboratory Shenzhen518055 China The State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing100191 China

We present an advanced study on more challenging high-resolution salient object detection (HRSOD) from both dataset and network framework perspectives. To compensate for the lack of HRSOD dataset, we thoughtfully collect a large-scale high resolution salient object detection dataset, called UHRSD, containing 5,920 images from real-world complex scenarios at 4K-8K resolutions. All the images are finely annotated in pixel-level, far exceeding previous low-resolution SOD datasets. Aiming at overcoming the contradiction between the sampling depth and the receptive field size in the past methods, we propose a novel one-stage framework for HR-SOD task using pyramid grafting mechanism. In general, transformer-based and CNN-based backbones are adopted to extract features from different resolution images independently and then these features are grafted from transformer branch to CNN branch. An attention-based Cross-Model Grafting Module (CMGM) is proposed to enable CNN branch to combine broken detailed information more holistically, guided by different source feature during decoding process. Moreover, we design an Attention Guided Loss (AGL) to explicitly supervise the attention matrix generated by CMGM to help the network better interact with the attention from different branches. Comprehensive experiments on UHRSD and widely-used SOD datasets demonstrate that our method can simultaneously locate salient object and preserve rich details, outperforming state-of-the-art methods. To verify the generalization ability of the proposed framework, we apply it to the camouflaged object detection (COD) task. Notably, our method performs superior to most state-of-the-art COD methods without bells and whistles. © 2024, CC BY.

关键词： Grafting (chemical)

Research on Crowdsourcing Price Game Model in Crowd Sensing

学校读者我要写书评

暂无评论

computers, Materials & Continua 2021年第8期68卷 1769-1784页

作者： Weijin Jiang Xiaoliang Liu Dejia Shi Junpeng Chen Yongxia Sun Liang Guo School of Computer Science and Technology Hunan University of Technology and BusinessChangsha410205China Key Laboratory of Hunan Province for New Retail Virtual Reality Technology Changsha410205China School of Bioinformatics University of MinnesotaTwin Cities15213USA

Crowd-Sensing is an innovative data acquisition method that combines the perception of mobile devices with the idea of *** is a new application mode under the development of the Internet of *** perceptual data that mobile users can provide is *** crowdsourcing parties will share this limited data,but the cost that the crowdsourcing party can pay is limited,and enough mobile users are needed to complete the perceptual task,making the group wisdom is really *** this process,there is bound to be a game between the crowds and the mobile *** of the existing researches consider a group-aware system.A group of mobile users will directly share or compete for the opportunity of the crowd-holders to do tasks and get paid,the behavior of multiple crowd-source parties,and their bilateral interaction with mobile *** research is not clear enough and there is no targeted *** paper will model and analyze the dynamic evolution process of crowd sensing *** on the unique characteristics of crowd-source non-cooperative game and crowd-sourced Nash equilibrium,we will develop a perceptual plan for mobile users and use the stability analysis of iterative algorithms to explore a way to better match the capabilities of mobile users and the needs of crowdsourced *** theoretical analysis and simulation results verify the dynamic evolution model of crowdsourcing in group perception and propose a method to improve the efficiency of crowdsourcing.

关键词： Crowd-sensing crowdsourcing incentives sensor networks

WinDB: HMD-free and Distortion-free Panoptic Video Fixation Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Guotao Chen, Chenglizhao Hao, Aimin Qin, Hong Fan, Deng-Ping The State Key Laboratory of Virtual Reality Technology and Systems Beihang University China The College of Computer Science and Technology China University of Petroleum China The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Research Unit of Virtual Human and Virtual Surgery Chinese Academy of Medical Sciences and with Pengcheng Laboratory China The Computer Science Department Stony Brook University United States ETH Zurich Zurich Switzerland

To date, the widely adopted way to perform fixation collection in panoptic video is based on a head-mounted display (HMD), where users' fixations are collected while wearing an HMD to explore the given panoptic scene freely. However, this widely-used data collection method is insufficient for training deep models to accurately predict which regions in a given panoptic are most important when it contains intermittent salient events. The main reason is that there always exist "blind zooms" when using HMD to collect fixations since the users cannot keep spinning their heads to explore the entire panoptic scene all the time. Consequently, the collected fixations tend to be trapped in some local views, leaving the remaining areas to be the "blind zooms". Therefore, fixation data collected using HMD-based methods that accumulate local views cannot accurately represent the overall global importance - the main purpose of fixations - of complex panoptic scenes. To conquer, this paper introduces the auxiliary window with a dynamic blurring (WinDB) fixation collection approach for panoptic video, which doesn't need HMD and is able to well reflect the regional-wise importance degree. Using our WinDB approach, we have released a new PanopticVideo-300 dataset, containing 300 panoptic clips covering over 225 categories. Specifically, since using WinDB to collect fixations is blind zoom free, there exists frequent and intensive "fixation shifting" - a very special phenomenon that has long been overlooked by the previous research - in our new set. Thus, we present an effective fixation shifting network (FishNet) to conquer it. All these new fixation collection tool, dataset, and network could be very potential to open a new age for fixation-related research and applications in 360o environments. © 2023, CC BY.

关键词： Helmet mounted displays

Exploring 3D Interaction with Gaze Guidance in Augmented Reality

学校读者我要写书评

暂无评论

Exploring 3D Interaction with Gaze Guidance in Augmented Rea...

IEEE Annual International Symposium virtual Reality

作者： Yiwei Bao Jiaxi Wang Zhimin Wang Feng Lu State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing China

Recent research based on hand-eye coordination has shown that gaze could improve object selection and translation experience under certain scenarios in AR. However, several limitations still exist. Specifically, we investigate whether gaze could help object selection with heavy 3D occlusions and help 3D object translation in the depth dimension. In addition, we also investigate the possibility of reducing the gaze calibration burden before use. Therefore, we develop new methods with proper gaze guidance for 3D interaction in AR, and also an implicit online calibration method. We conduct two user studies to evaluate different interaction methods and the results show that our methods not only improve the effectiveness of occluded objects selection but also alleviate the arm fatigue problem significantly in the depth translation task. We also evaluate the proposed implicit online calibration method and find its accuracy comparable to standard 9 points explicit calibration, which makes a step towards practical use in the real world.

关键词： Three-dimensional displays Design methodology Estimation User interfaces Fatigue Calibration Task analysis

SVDFed: Enabling Communication-Efficient Federated Learning via Singular-Value-Decomposition 42

学校读者我要写书评

暂无评论

SVDFed: Enabling Communication-Efficient Federated Learning ...

42nd IEEE International Conference on computer Communications, INFOCOM 2023

作者： Wang, Haolin Liu, Xuefeng Niu, Jianwei Tang, Shaojie Beihang University State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beijing China Zhongguancun Laboratory Beijing China Zhengzhou University Zhengzhou University Research Institute of Industrial Technology School of Information Engineering Zhengzhou China Naveen Jindal School Management University of Texas at Dallas Richardson United States

ISBN: (纸本)9798350334142

Federated learning (FL) is an emerging paradigm of distributed machine learning. However, when applied to wireless network scenarios, FL usually suffers from high communication cost because clients need to transmit their updated gradients to a server in every training round. Although many gradient compression techniques like sparsification and quantization are proposed, they compress clients' gradients independently, without considering the correlations among gradients. In this paper, we propose SVDFed, a collaborative gradient compression framework for FL. SVDFed utilizes Singular Value Decomposition (SVD) to find a few basis vectors, whose linear combination can well represent clients' gradients at a certain round. Due to the correlations among gradients, these basis vectors can still well approximate new gradients in many subsequent rounds. With the help of basis vectors, clients only need to upload the coefficients of the linear combination to the server, which greatly reduces communication cost. In addition, SVDFed leverages the classical PID (Proportional, Integral, Derivative) control to determine the proper time to update basis vectors to maintain their representation ability. Through experiments, we demonstrate that SVDFed outperforms existing gradient compression methods in FL. For example, compared to a popular gradient quantization method QSGD, SVDFed can reduce the communication overhead by 66 % and pending time by 99 %. © 2023 IEEE.

关键词： Singular value decomposition

Depression Scale Dictionary Decomposition Framework for Multimodal Automatic Depression Level Prediction

学校读者我要写书评

暂无评论

IEEE Transactions on Circuits and Systems for Video technology 2025年第6期35卷 6195-6210页

作者： Niu, Mingyue Wang, Xu Gong, Jibing Liu, Bin Tao, Jianhua Schuller, Bjorn W. Yanshan University School of Information Science and Engineering Hebei province Qinhuangdao066004 China Hebei Key Laboratory of Computer Virtual Technology and System Integration China Technology Co. Ltd Bejing100022 China Bejing100190 China Tsinghua University Department of Automation Bejing100190 China University of Chinese Academy of Sciences School of Artificial Intelligence Beijing100049 China University of Augsburg ZD.B. Chair of Embedded Intelligence for Health Care and Wellbeing Augsburg Germany Glam - Group on Language Audio and Music United Kingdom

Currently, many researchers aim to achieve automatic depression level prediction via speech and video behavior analysis. However, previous works have struggled to decompose audio and video sequences into the information related to and unrelated to depression scores, hindering the model's perception of depression cues. Besides, previous works implement multimodal fusion using attention mechanisms or linear layers, but failed to simultaneously consider the Euclidean relationship among tokens and the non-Euclidean relationship among channels, which bring limitations in capturing depression cues. In response to the above issues, we propose a depression scale dictionary decomposition framework, which mainly includes a Bidirectional Dictionary Decomposition (BDD) module and a Bidirectional Multimodal Fusion (BMF) module. The BDD module can use the dictionaries generated based on the depression scale to semantically decompose audio and video sequences into the information related to and unrelated to depression scores along token and channel dimensions for promoting depression cue perception. Moreover, considering the respective characteristics of tokens and channels, the BMF module uses linear layers and graph convolution to achieve cross-modal mixing, which is used to aggregate audio and video sequences for predicting depression levels. The validation on AVEC 2013, AVEC 2014 and DAIC-WOZ datasets demonstrates our method's superiority. © 1991-2012 IEEE.

关键词： Boolean functions

MLLMReID: Multimodal Large Language Model-based Person Re-identification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Shan Zhang, Yongfei Beijing Key Laboratory of Digital Media School of Computer Science and Engineering Beihang University Beijing100191 China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China

Multimodal large language models (MLLM) have achieved satisfactory results in many tasks. However, their performance in the task of ReID (ReID) has not been explored to date. This paper will investigate how to adapt them for the task of ReID. An intuitive idea is to fine-tune MLLM with ReID image-text datasets, and then use their visual encoder as a backbone for ReID. However, there still exist two apparent issues: (1) Designing instructions for ReID, MLLMs may overfit specific instructions, and designing a variety of instructions will lead to higher costs. (2) When fine-tuning the visual encoder of a MLLM, it is not trained synchronously with the ReID task. As a result, the effectiveness of the visual encoder fine-tuning cannot be directly reflected in the performance of the ReID task. To address these problems, this paper proposes MLLMReID: Multimodal Large Language Model-based ReID. Firstly, we proposed Common Instruction, a simple approach that leverages the essence ability of LLMs to continue writing, avoiding complex and diverse instruction design. Secondly, we propose a multi-task learning-based synchronization module to ensure that the visual encoder of the MLLM is trained synchronously with the ReID task. The experimental results demonstrate the superiority of our method. © 2024, CC BY-NC-ND.

关键词： Learning systems

Locomotion perception and redirection

学校读者我要写书评

暂无评论

virtual Reality & Intelligent Hardware 2021年第6期3卷 I0003-I0004页

作者： Miao WANG Songhai ZHANG Shimin HU State Key Laboratory of Virtual Reality Technology and Systems Beihang UniversityBeijing 100191China Department of Computer Science and Technology Tsinghua UniversityBeijing 100084China

Locomotion is a fundamental interaction technique that allows free navigation in virtual scenes.A large body of literature has demonstrated that natural locomotion experience can significantly improve the sense of presence and help users achieve better performance in tasks such as searching and *** the upcoming metaverse era,locomotion will play a critical role in the success of large-scale interactive *** and redirection are two tightly connected problems in virtual *** a limited physical condition and a target virtual scene,how to make the perceived locomotion as natural as possible,and further unknowingly redirect the user in the physical world is an open *** this special issue,we have selected five papers that provide their latest updates for research problems of view control,virtual jumping,climbing,and searching.

关键词： redirect interactive motion