检索结果-内蒙古大学图书馆

AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Baisong Wang, Xingwang Xu, Haixiao School of Computer Science and Technology Jilin University China Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University China

Large language models(LLMs) excellent performance across a variety of tasks, but they come with significant computational and storage costs. Quantizing these models is an effective way to alleviate this issue. However, existing methods struggle to strike a balance between model accuracy and hardware efficiency. This is where we introduce AWEQ, a post-training method that requires no additional training overhead. AWEQ excels in both ultra-low-bit quantization and 8-bit weight and activation (W8A8) quantization. There is an observation that weight quantization is less challenging than activation quantization. AWEQ transfers the difficulty of activation quantization to weights using channel equalization, achieving a balance between the quantization difficulties of both, and thereby maximizing performance. We have further refined the equalization method to mitigate quantization bias error, ensuring the robustness of the model. Extensive experiments on popular models such as LLaMA and OPT demonstrate that AWEQ outperforms all existing post-training quantization methods for large models. Copyright © 2023, The Authors. All rights reserved.

关键词： Chemical activation

Learning Interpretable Network Dynamics via Universal Neural symbolic Regression

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hu, Jiao Cui, Jiaxu Yang, Bo College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China

Discovering governing equations of complex network dynamics is a fundamental challenge in contemporary science with rich data, which can uncover the mysterious patterns and mechanisms of the formation and evolution of complex phenomena in various fields and assist in decision-making. In this work, we develop a universal computational tool that can automatically, efficiently, and accurately learn the symbolic changing patterns of complex system states by combining the excellent fitting ability from deep learning and the equation inference ability from pre-trained symbolic regression. We conduct intensive experimental verifications on more than ten representative scenarios from physics, biochemistry, ecology, epidemiology, etc. Results demonstrate the outstanding effectiveness and efficiency of our tool by comparing with the state-of-the-art symbolic regression techniques for network dynamics. The application to real-world systems including global epidemic transmission and pedestrian movements has verified its practical applicability. We believe that our tool can serve as a universal solution to dispel the fog of hidden mechanisms of changes in complex phenomena, advance toward interpretability, and inspire more scientific discoveries. Copyright © 2024, The Authors. All rights reserved.

关键词： Economic and social effects

AHMSA-Net: Adaptive Hierarchical Multi-Scale Attention Network for Micro-Expression Recognition

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Lijun Zhang, Yifan Tang, Weicheng Sun, Xinzhi Wang, Xiaomeng Li, Zhanshan College of Computer Science and Technology Jilin University Changchun Jilin130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun Jilin130012 China

Micro-expression recognition (MER) presents a significant challenge due to the transient and subtle nature of the motion changes involved. In recent years, deep learning methods based on attention mechanisms have made some breakthroughs in MER. However, these methods still suffer from the limitations of insufficient feature capture and poor dynamic adaptation when coping with the instantaneous subtle movement changes of micro-expressions. Therefore, in this paper, we design an Adaptive Hierarchical Multi-Scale Attention Network (AHMSA-Net) for MER. Specifically, we first utilize the onset and apex frames of the micro-expression sequence to extract threedimensional (3D) optical flow maps, including horizontal optical flow, vertical optical flow, and optical flow strain. Subsequently, the optical flow feature maps are inputted into AHMSA-Net, which consists of two parts: an adaptive hierarchical framework and a multi-scale attention mechanism. Based on the adaptive downsampling hierarchical attention framework, AHMSA-Net captures the subtle changes of micro-expressions from different granularities (fine and coarse) by dynamically adjusting the size of the optical flow feature map at each layer. Based on the multi-scale attention mechanism, AHMSA-Net learns micro-expression action information by fusing features from different scales (channel and spatial). These two modules work together to comprehensively improve the accuracy of MER. Additionally, rigorous experiments demonstrate that the proposed method achieves competitive results on major micro-expression databases, with AHMSA-Net achieving recognition accuracy of up to 78.21% on composite databases (SMIC, SAMM, CASMEII) and 77.08% on the CASME3 database. Copyright © 2025, The Authors. All rights reserved.

关键词： Deep learning

Enhancing Unsupervised Graph Few-shot Learning via Set Functions and Optimal Transport

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Liu, Yonghao Giunchiglia, Fausto Li, Ximing Huang, Lan Feng, Xiaoyue Guan, Renchu College of Computer Science and Technology Jilin University Changchun China Department of Information Engineering and Computer Science University of Trento Trento Italy Key Laboratory of Symbolic Computation and Knowledge Engineering The Ministry of Education China

Graph few-shot learning has garnered significant attention for its ability to rapidly adapt to downstream tasks with limited labeled data, sparking considerable interest among researchers. Recent advancements in graph few-shot learning models have exhibited superior performance across diverse applications. Despite their successes, several limitations still exist. First, existing models in the meta-training phase predominantly focus on instance-level features within tasks, neglecting crucial set-level features essential for distinguishing between different categories. Second, these models often utilize query sets directly on classifiers trained with support sets containing only a few labeled examples, overlooking potential distribution shifts between these sets and leading to suboptimal performance. Finally, previous models typically require necessitate abundant labeled data from base classes to extract transferable knowledge, which is typically infeasible in real-world scenarios. To address these issues, we propose a novel model named STAR, which leverages Set funcTions and optimAl tRansport for enhancing unsupervised graph few-shot learning. Specifically, STAR utilizes expressive set functions to obtain set-level features in an unsupervised manner and employs optimal transport principles to align the distributions of support and query sets, thereby mitigating distribution shift effects. Theoretical analysis demonstrates that STAR can capture more task-relevant information and enhance generalization capabilities. Empirically, extensive experiments across multiple datasets validate the effectiveness of STAR. Our code can be found here. Copyright © 2025, The Authors. All rights reserved.

关键词： Zero-shot learning

Verifying Diagnosability of Discrete Event System with Logical Formula

学校读者我要写书评

暂无评论

Chinese Journal of Electronics 2020年第2期29卷 304-311页

作者： GENG Xuena OUYANG Dantong HAN Cheng College of Computer Science and Technology Changchun University of Science and Technology Key Laboratory of Symbolic Computation and Knowledge Engineering for Ministry of Education Jilin University

Diagnosability is an important property in the field of fault diagnosis. In this paper, a novel approach based on logical formula is proposed to verify diagnosability of Discrete event systems(DESs). CNFFSM is defined to represent a new model for DES. Each transition in DES can be described as a clause. According to CNF-FSM, we construct a CNF-diagnoser. Based on the resolution principle and CNF-diagnoser, an algorithm is presented to test whether the failure events can be detected or not in a finite number of observable *** algorithm can be applied in both off-line diagnosis and on-line diagnosis. Experimental results show that our algorithm can solve the diagnosability problem efficiently.

关键词： Fault diagnosis Diagnosability Discrete event system Finite state machine Conjunctive normal form

LLM-Powered User Simulator for Recommender System 39

学校读者我要写书评

暂无评论

LLM-Powered User Simulator for Recommender System

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Zhang, Zijian Liu, Shuchang Liu, Ziru Zhong, Rui Cai, Qingpeng Zhao, Xiangyu Zhang, Chunxu Liu, Qidong Jiang, Peng Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University China Kuaishou Technology China City University of Hong Kong Hong Kong Xi'an Jiaotong University China

ISBN: (纸本)157735897X

User simulators can rapidly generate a large volume of timely user behavior data, providing a testing platform for reinforcement learning-based recommender systems, thus accelerating their iteration and optimization. However, prevalent user simulators generally suffer from significant limitations, including the opacity of user preference modeling and the incapability of evaluating simulation accuracy. In this paper, we introduce an LLM-powered user simulator to simulate user engagement with items in an explicit manner, thereby enhancing the efficiency and effectiveness of reinforcement learning-based recommender systems training. Specifically, we identify the explicit logic of user preferences, leverage LLMs to analyze item characteristics and distill user sentiments, and design a logical model to imitate real human engagement. By integrating a statistical model, we further enhance the reliability of the simulation, proposing an ensemble model that synergizes logical and statistical insights for user interaction simulations. Capitalizing on the extensive knowledge and semantic generation capabilities of LLMs, our user simulator faithfully emulates user behaviors and preferences, yielding high-fidelity training data that enrich the training of recommendation algorithms. We establish quantifying and qualifying experiments on five datasets to validate the simulator's effectiveness and stability across various recommendation scenarios. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词： Reinforcement learning

Computing PUR of Zero-Dimensional Ideals of Breadth at Most One

学校读者我要写书评

暂无评论

Journal of Systems Science & Complexity 2021年第6期34卷 2396-2409页

作者： PAN Jian SHANG Baoxin LI Zhe ZHANG Shugong School of Mathematics Key Laboratory of Symbolic Computation and Knowledge Engineering(Ministry of Education)Jilin UniversityChangchun 130012China College of Science Northeast Electric Power UniversityJilin 132012China School of Science Changchun University of Science and TechnologyChangchun 130022China

In this paper,for a zero-dimensional polynomial ideal I,the authors prove that k[x_(1),x_(2),…,x_(n)]/I is cyclic if and only if the breadth of I is 0 or ***,the authors present a new algorithm to compute polynomial ... 详细信息

关键词： Cyclic basis Grobner basis polynomial univariate representation separating element

Skeleton-based Action Recognition with Non-linear Dependency Modeling and Hilbert-Schmidt Independence Criterion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Haipeng Yang, Yuheng Lyu, Yingda College of Computer Science and Technology Jilin University China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University China Public Computer Education and Research Center Jilin University China

Human skeleton-based action recognition has long been an indispensable aspect of artificial intelligence. Current state-of-the-art methods tend to consider only the dependencies between connected skeletal joints, limiting their ability to capture non-linear dependencies between physically distant joints. Moreover, most existing approaches distinguish action classes by estimating the probability density of motion representations, yet the high-dimensional nature of human motions invokes inherent difficulties in accomplishing such measurements. In this paper, we seek to tackle these challenges from two directions: (1) We propose a novel dependency refinement approach that explicitly models dependencies between any pair of joints, effectively transcending the limitations imposed by joint distance. (2) We further propose a framework that utilizes the Hilbert-Schmidt Independence Criterion to differentiate action classes without being affected by data dimensionality, and mathematically derive learning objectives guaranteeing precise recognition. Empirically, our approach sets the state-of-the-art performance on NTU RGB+D, NTU RGB+D 120, and Northwestern-UCLA datasets. © 2024, CC BY-SA.

关键词： Probability density function

Dual-level Mixup for Graph Few-shot Learning with Fewer Tasks

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Liu, Yonghao Li, Mengyu Giunchiglia, Fausto Huang, Lan Li, Ximing Feng, Xiaoyue Guan, Renchu College of Computer Science and Technology Jilin University Changchun China Department of Information Engineering and Computer Science University of Trento Trento Italy Key Laboratory of Symbolic Computation and Knowledge Engineering The Ministry of Education China

Graph neural networks have been demonstrated as a powerful paradigm for effectively learning graph-structured data on the web and mining content from it. Current leading graph models require a large number of labeled samples for training, which unavoidably leads to overfitting in few-shot scenarios. Recent research has sought to alleviate this issue by simultaneously leveraging graph learning and meta-learning paradigms. However, these graph meta-learning models assume the availability of numerous meta-training tasks to learn transferable meta-knowledge. Such assumption may not be feasible in the real world due to the difficulty of constructing tasks and the substantial costs involved. Therefore, we propose a SiMple yet effectIve approach for graph few-shot Learning with fEwer tasks, named SMILE. We introduce a dual-level mixup strategy, encompassing both within-task and across-task mixup, to simultaneously enrich the available nodes and tasks in meta-learning. Moreover, we explicitly leverage the prior information provided by the node degrees in the graph to encode expressive node representations. Theoretically, we demonstrate that SMILE can enhance the model generalization ability. Empirically, SMILE consistently outperforms other competitive models by a large margin across all evaluated datasets with in-domain and cross-domain settings. Our anonymous code can be found here. Copyright © 2025, The Authors. All rights reserved.

关键词： Contrastive Learning