检索结果-内蒙古大学图书馆

Adaptive data-Free Quantization

学校读者我要写书评

暂无评论

arXiv 2023年

data-free quantization (DFQ) recovers the performance of quantized network (Q) without accessing the original data, but generates the fake sample via a generator (G) by learning from full-precision network (P), which, however, is totally independent of Q, overlooking the adaptability of the knowledge from generated samples, i.e., informative or not to the learning process of Q, resulting into the overflow of generalization error. Building on this, several critical questions - how to measure the sample adaptability to Q under varied bit-width scenarios? how to generate the samples with large adaptability to improve Q's generalization? whether the largest adaptability is the best? To answer the above questions, in this paper, we propose an Adaptive data-Free Quantization (AdaDFQ) method, which revisits DFQ from a zero-sum game perspective upon the sample adaptability between two players - a generator and a quantized network. Following this viewpoint, we further define the disagreement and agreement samples to form two boundaries, where the margin is optimized to address the over-and-under fitting issues, so as to generate the samples with adaptive adaptability to Q. Our AdaDFQ reveals: 1) the largest adaptability is NOT the best for sample generation to benefit Q's generalization;2) the knowledge of the generated sample should not be informative to Q only, but also related to the category and distribution information of the training data for P. The theoretical and empirical analysis validate the advantages of AdaDFQ over the state-of-the-arts. Our code is available at https://***/hfutqian/AdaDFQ. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

EdgeNN: Efficient Neural Network Inference for CPU-GPU Integrated Edge Devices

学校读者我要写书评

暂无评论

EdgeNN: Efficient Neural Network Inference for CPU-GPU Integ...

International Conference on data engineering

作者： Chenyang Zhang Feng Zhang Kuangyu Chen Mingjun Chen Bingsheng He Xiaoyong Du Key Laboratory of Data Engineering and Knowledge Engineering (MOE) and School of Information Renmin University of China School of Computing National University of Singapore

With the development of the architectures and the growth of AIoT application requirements, data processing on edge has become popular. Neural network inference is widely employed for data analytics on edge devices. This paper extensively explores neural network inference on integrated edge devices and proposes EdgeNN, the first neural network inference solution on CPU-GPU integrated edge devices. EdgeNN has three novel characteristics. First, EdgeNN can adaptively utilize the unified physical memory and conduct the zero-copy optimization. Second, EdgeNN involves a novel inference-targeted inter- and intra-kernel CPU-GPU hybrid execution approach, which co-runs the CPU with the GPU to fully utilize the edge device’s computing resources. Third, EdgeNN adopts a fine-grained adaptive inference tuning approach, which can divide the complicated inference structure into sub-tasks mapped to the CPU and the GPU. Experiments show that on six popular neural network inference tasks, EdgeNN brings an average of 3.97×, 3.12×, and 8.80× speedups to inference on the CPU of the integrated device, inference on a mobile phone CPU, and inference on an edge CPU device. Additionally, it achieves 22.02% time benefits to the direct execution of the original programs. Specifically, 9.93% comes from better utilization of unified memory, and 10.76% comes from CPU-GPU hybrid execution. Besides, EdgeNN can deliver 29.14× and 5.70× higher energy efficiency than the edge CPU and the discrete GPU, respectively. We have made EdgeNN available at https://***/ChenyangZhang-cs/EdgeNN.

关键词：

An Actor-centric Causality Graph for Asynchronous Temporal Inference in Group Activity

学校读者我要写书评

暂无评论

An Actor-centric Causality Graph for Asynchronous Temporal I...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Zhao Xie Tian Gao Kewei Wu Jiao Chang Key Laboratory of Knowledge Engineering with Big Data Hefei University of Technology School of Computer Science and Information Engineering Hefei University of Technology

The causality relation modeling remains a challenging task for group activity recognition. The causality relations describe the influence on the centric actor (effect actor) from its correlative actors (cause actors). Most existing graph models focus on learning the actor relation with synchronous temporal features, which is insufficient to deal with the causality relation with asynchronous temporal features. In this paper, we propose an Actor-Centric Causality Graph Model, which learns the asynchronous temporal causality relation with three modules, i.e., an asynchronous temporal causality relation detection module, a causality feature fusion module, and a causality relation graph inference module. First, given a centric actor and its correlative actor, we analyze their influences to detect causality relation. We estimate the self influence of the centric actor with self regression. We estimate the correlative influence from the correlative actor to the centric actor with correlative regression, which uses asynchronous features at different timestamps. Second, we synchronize the two action features by estimating the temporal delay between the cause action and the effect action. The synchronized features are used to enhance the feature of the effect action with a channel-wise fusion. Third, we describe the nodes (actors) with causality features and learn the edges by fusing the causality relation with the appearance relation and distance relation. The causality relation graph inference provides crucial features of effect action, which are complementary to the base model using synchronous relation inference. Experiments show that our method achieves state-of-the-art performance on the Volleyball dataset and Collective Activity dataset.

关键词：

Adaptive data-Free Quantization

学校读者我要写书评

暂无评论

Adaptive Data-Free Quantization

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Biao Qian Yang Wang Richang Hong Meng Wang Key Laboratory of Knowledge Engineering with Big Data Ministry of Education School of Computer Science and Information Engineering Hefei University of Technology China

data-free quantization (DFQ) recovers the performance of quantized network (Q) without the original data, but generates the fake sample via a generator (G) by learning from full-precision network (P), which, however, is totally independent of Q, overlooking the adaptability of the knowledge from generated samples, i.e., informative or not to the learning process of Q, resulting into the overflow of generalization error. Building on this, several critical questions — how to measure the sample adaptability to Q under varied bit-width scenarios? whether the largest adaptability is the best? how to generate the samples with adaptive adaptability to improve Q's generalization? To answer the above questions, in this paper, we propose an Adaptive data-Free Quantization (AdaDFQ) method, which revisits DFQ from a zero-sum game perspective upon the sample adaptability between two players — a generator and a quantized network. Following this viewpoint, we further define the disagreement and agreement samples to form two boundaries, where the margin between two boundaries is optimized to adaptively regulate the adaptability of generated samples to Q, so as to address the over-and-under fitting issues. Our AdaDFQ reveals: 1) the largest adaptability is NOT the best for sample generation to benefit Q's generalization; 2) the knowledge of the generated sample should not be informative to Q only, but also related to the category and distribution information of the training data for P. The theoretical and empirical analysis validate the advantages of AdaDFQ over the state-of-the-arts. Our code is available at https://***/hfutqian/AdaDFQ.

关键词：

LLMIdxAdvis: Resource-Efficient Index Advisor Utilizing Large Language Model

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhao, Xinxin Li, Haoyang Zhang, Jing Huang, Xinmei Zhang, Tieying Chen, Jianjun Shi, Rui Li, Cuiping Chen, Hong School of Information Renmin University of China Beijing China Key Laboratory of Data Engineering and Knowledge Engineering MOE China Engineering Research Center of Database and Business Intelligence MOE China ByteDance China

Index recommendation is essential for improving query performance in database management systems (DBMSs) through creating an optimal set of indexes under specific constraints. Traditional methods, such as heuristic and learning-based approaches, are effective but face challenges like lengthy recommendation time, resource-intensive training, and poor generalization across different workloads and database schemas. To address these issues, we propose LLMIdxAdvis, a resource-efficient index advisor that uses large language models (LLMs) without extensive fine-tuning. LLMIdxAdvis frames index recommendation as a sequence-to-sequence task, taking target workload, storage constraint, and corresponding database environment as input, and directly outputting recommended indexes. It constructs a high-quality demonstration pool offline, using GPT-4-Turbo to synthesize diverse SQL queries and applying integrated heuristic methods to collect both default and refined labels. During recommendation, these demonstrations are ranked to inject database expertise via in-context learning. Additionally, LLMIdxAdvis extracts workload features involving specific column statistical information to strengthen LLM’s understanding, and introduces a novel inference scaling strategy combining vertical scaling (via "Index-Guided Major Voting" and Best-of-N) and horizontal scaling (through iterative "self-optimization" with database feedback) to enhance reliability. Experiments on 3 OLAP and 2 real-world benchmarks reveal that LLMIdxAdvis delivers competitive index recommendation with reduced runtime, and generalizes effectively across different workloads and database schemas. Copyright © 2025, The Authors. All rights reserved.

关键词： Query languages

Generative Zero-shot Learning Compound Fault Diagnosis of Bearings 2

学校读者我要写书评

暂无评论

Generative Zero-shot Learning Compound Fault Diagnosis of Be...

2nd International Conference on Sensing, Measurement and data Analytics in the Era of Artificial Intelligence, ICSMD 2021

作者： Xu, Juan Li, Kang Hefei University of Technology School of Computer and Information Key Laboratory of Knowledge Engineering with Big Data Hefei China

ISBN: (纸本)9781665427470

Diagnosis of compound faults remains a challenge during fault diagnosis of bearings, owing to the different fault parameters coupling, fault characteristics diversity, and the exponential increasement of the number of possible failure modes. Current compound faults diagnostic methods, which are usually based on supervised or semi-supervised learning, require sufficient labeled or unlabeled training data for each compound faults. In industrial scenarios, neither labeled nor unlabeled training data of compound faults are usually difficult to collect and sometimes even inaccessible, whereas single faults samples are easy to obtain. Based on these issues, we construct a novel generative zero-shot learning (ZSL) compound faults diagnosis model identifies unseen compound faults using only single faults samples as training set. This model comprises several modules, namely semantic vector definition, feature extractor, generative adversarial modules. Firstly, we devise a unified semantic vector definition method for expressing single and compound faults based on theoretical correlation of characteristics between single fault and compound faults vibration data. Secondly, a CNN-based feature extractor is designed for extraction the fault features from the time-frequency domain of vibration data. Thirdly, a generative adversarial module performs adversarial training of semantic vectors and fault features of single faults to learn the mapping relationship between the fault features and the fault semantic vectors. Once trained, the generator is able to generate compound fault features using the compound fault semantic vectors, rather than any compound fault samples. Finally, the K-nearest neighbor method is adopted to identify the unseen compound faults by measuring the distance between the extracted feature from the testing compound fault samples and the generated features. The effectiveness of the proposed method is verified on a self-built bearing test stand. The results show

关键词： Semantics

TABLELLM: Enabling Tabular data Manipulation by LLMs in Real Office Usage Scenarios

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Xiaokang Luo, Sijia Zhang, Bohan Ma, Zeyao Zhang, Jing Li, Yang Li, Guanlin Yao, Zijun Xu, Kangli Zhou, Jinchang Zhang-Li, Daniel Yu, Jifan Zhao, Shu Li, Juanzi Tang, Jie School of Information Renmin University of China China Tsinghua University China Key Laboratory of Data Engineering and Knowledge Engineering Beijing China Computer Science Anhui University China

We introduce TABLELLM, a robust large language model (LLM) with 8 billion parameters, purpose-built for proficiently handling tabular data manipulation tasks, whether they are embedded within documents or spreadsheets, catering to real-world office scenarios. We propose a distant supervision method for training, which comprises a reasoning process extension strategy, aiding in training LLMs to understand reasoning patterns more effectively as well as a cross-way validation strategy, ensuring the quality of the automatically generated data. To evaluate the performance of TABLELLM, we have crafted benchmarks tailored to address both document and spreadsheet formats as well as constructed a well-organized evaluation pipeline capable of handling both scenarios. Thorough evaluations underscore the advantages of TABLELLM when compared to various existing general-purpose and tabular data-focused LLMs. We have publicly released the model checkpoint, source code, benchmarks, and a web application for user interaction. Copyright © 2024, The Authors. All rights reserved.

关键词： Spreadsheets

WATuning:A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2021年第4期36卷 741-761页

作者： Jia-Ke Ge Yan-Feng Chai Yun-Peng Chai Key Laboratory of Data Engineering and Knowledge Engineering of Ministry of Education Renmin University of China Beijing 100872China School of Information Renmin University of ChinaBeijing 100872China College of Computer Science and Technology Taiyuan University of Science and TechnologyTaiyuan 030027China

Configuration tuning is essential to optimize the performance of systems(e.g.,databases,key-value stores).High performance usually indicates high throughput and low *** present,most of the tuning tasks of systems are performed artificially(e.g.,by database administrators),but it is hard for them to achieve high performance through tuning in various types of systems and in various *** recent years,there have been some studies on tuning traditional database systems,but all these methods have some *** this article,we put forward a tuning system based on attention-based deep reinforcement learning named WATuning,which can adapt to the changes of workload characteristics and optimize the system performance efficiently and ***,we design the core algorithm named ATT-Tune for WATuning to achieve the tuning task of *** algorithm uses workload characteristics to generate a weight matrix and acts on the internal metrics of systems,and then ATT-Tune uses the internal metrics with weight values assigned to select the appropriate ***,WATuning can generate multiple instance models according to the change of the workload so that it can complete targeted recommendation services for different types of ***,WATuning can also dynamically fine-tune itself according to the constantly changing workload in practical applications so that it can better fit to the actual environment to make *** experimental results show that the throughput and the latency of WATuning are improved by 52.6%and decreased by 31%,respectively,compared with the throughput and the latency of CDBTune which is an existing optimal tuning method.

关键词： attention mechanism auto-tuning system reinforcement learning(RL) workload-aware

PathReasoner: Modeling Reasoning Path with Equivalent Extension for Logical Question Answering

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xu, Fangzhi Lin, Qika Zhao, Tianzhe Han, Jiawei Liu, Jun School of Computer Science and Technology Xi’an Jiaotong University China Ministry of Education Key Laboratory of Intelligent Networks and Network Security China Shaanxi Province Key Laboratory of Big Data Knowledge Engineering China

Logical reasoning task has attracted great interest since it was proposed. Faced with such a task, current competitive models, even large language models (e.g., ChatGPT and PaLM 2), still perform badly. Previous promising LMs struggle in logical consistency modeling and logical structure perception. To this end, we model the logical reasoning task by transforming each logical sample into reasoning paths and propose an architecture PathReasoner. It addresses the task from the views of both data and model. To expand the diversity of the logical samples, we propose an atom extension strategy supported by equivalent logical formulas, to form new reasoning paths. From the model perspective, we design a stack of transformer-style blocks. In particular, we propose a path-attention module to joint model in-atom and cross-atom relations with the high-order diffusion strategy. Experiments show that PathReasoner achieves competitive performances on two logical reasoning benchmarks and great generalization abilities. © 2024, CC BY.

关键词： Atoms