检索结果-内蒙古大学图书馆

arXiv 2025年

作者： Bi, Zhangqian Wan, Yao Chu, Zhaoyang Hu, Yufei Zhang, Junyi Zhang, Hongyu Xu, Guandong Jin, Hai National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab Wuhan China School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China School of Big Data and Software Engineering Chongqing University Chongqing China School of Computer Science University of Technology Sydney Sydney Australia

Pre-training a language model and then fine-tuning it has shown to be an efficient and effective technique for a wide range of code intelligence tasks, such as code generation, code summarization, and vulnerability detection. However, pre-training language models on a large-scale code corpus is computationally expensive. Fortunately, many off-the-shelf Pre-trained Code Models (PCMs), such as CodeBERT, CodeT5, CodeGen, and Code Llama, have been released publicly. These models acquire general code understanding and generation capability during pre-training, which enhances their performance on downstream code intelligence tasks. With an increasing number of these public pre-trained models, selecting the most suitable one to reuse for a specific task is essential. In this paper, we systematically investigate the reusability of PCMs. We first explore three intuitive model selection methods that select by size, training data, or brute-force fine-tuning. Experimental results show that these straightforward techniques either perform poorly or suffer high costs. Motivated by these findings, we explore learning-based model selection strategies that utilize pre-trained models without altering their parameters. Specifically, we train proxy models to gauge the performance of pre-trained models, and measure the distribution deviation between a model's latent features and the task's labels, using their closeness as an indicator of model transferability. We conduct experiments on 100 widely-used open-source PCMs for code intelligence tasks, with sizes ranging from 42.5 million to 3 billion parameters. The results demonstrate that learning-based selection methods reduce selection time to 100 seconds, compared to 2,700 hours with brute-force fine-tuning, with less than 6% performance degradation across related tasks. © 2025, CC BY-NC-SA.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Study

arXiv

引用

arXiv 2024年

作者： Wu, Yang Wan, Yao Zhang, Hongyu Sui, Yulei Wei, Wucai Zhao, Wei Xu, Guandong Jin, Hai Huazhong University of Science and Technology China Chongqing University China University of New South Wales Australia University of Technology Sydney Australia National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

The Natural Language to Visualization (NL2Vis) task aims to transform natural-language descriptions into visual representations for a grounded table, enabling users to gain insights from vast amounts of data. Recently, many deep learning-based approaches have been developed for NL2Vis. Despite the considerable efforts made by these approaches, challenges persist in visualizing data sourced from unseen databases or spanning multiple tables. Taking inspiration from the remarkable generation capabilities of Large Language Models (LLMs), this paper conducts an empirical study to evaluate their potential in generating visualizations, and explore the effectiveness of in-context learning prompts for enhancing this task. In particular, we first explore the ways of transforming structured tabular data into sequential text prompts, as to feed them into LLMs and analyze which table content contributes most to the NL2Vis. Our findings suggest that transforming structured tabular data into programs is effective, and it is essential to consider the table schema when formulating prompts. Furthermore, we evaluate two types of LLMs: finetuned models (e.g., T5-Small) and inference-only models (e.g., GPT-3.5), against state-of-the-art methods, using the NL2Vis benchmarks (i.e., nvBench). The experimental results reveal that LLMs outperform baselines, with inference-only models consistently exhibiting performance improvements, at times even surpassing fine-tuned models when provided with certain few-shot demonstrations through in-context learning. Finally, we analyze when the LLMs fail in NL2Vis, and propose to iteratively update the results using strategies such as chain-of-thought, role-playing, and code-interpreter. The experimental results confirm the efficacy of iterative updates and hold great potential for future study. © 2024, CC BY.

关键词： Data visualization

来源：评论

学校读者我要写书评

暂无评论

OCDB: REVISITING CAUSAL DISCOVERY WITH A COMPREHENSIVE BENCHMARK AND EVALUATION FRAMEWORK

arXiv

引用

arXiv 2024年

作者： Zhou, Wei Huang, Hong Zhang, Guowen Shi, Ruize Yin, Kehan Lin, Yuanyuan Liu, Bang Huazhong University of Science and Technology China DIRO Université de Montréal & Mila Canada CIFAR AI Chair Canada The National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

Large language models (LLMs) have excelled in various natural language processing tasks, but challenges in interpretability and trustworthiness persist, limiting their use in high-stakes fields. Causal discovery offers a promising approach to improve transparency and reliability. However, current evaluations are often one-sided and lack assessments focused on interpretability performance. Additionally, these evaluations rely on synthetic data and lack comprehensive assessments of real-world datasets. These lead to promising methods potentially being overlooked. To address these issues, we propose a flexible evaluation framework with metrics for evaluating differences in causal structures and causal effects, which are crucial attributes that help improve the interpretability of LLMs. We introduce the Open Causal Discovery Benchmark (OCDB), based on real data, to promote fair comparisons and drive optimization of algorithms. Additionally, our new metrics account for undirected edges, enabling fair comparisons between Directed Acyclic Graphs (DAGs) and Completed Partially Directed Acyclic Graphs (CPDAGs). Experimental results show significant shortcomings in existing algorithms' generalization capabilities on real data, highlighting the potential for performance improvement and the importance of our framework in advancing causal discovery techniques. Copyright © 2024, The Authors. All rights reserved.

关键词： Directed graphs

来源：评论

学校读者我要写书评

暂无评论

CPSAA: Accelerating Sparse Attention using Crossbar-based Processing-In-Memory Architecture

arXiv

引用

arXiv 2022年

作者： Li, Huize Jin, Hai Zheng, Long Liao, Xiaofei Huang, Yu Liu, Cong Xu, Jiahong Duan, Zhuohui Chen, Dan Gui, Chuangyi National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China

The attention-based neural network attracts great interest due to its excellent accuracy enhancement. However, the attention mechanism requires huge computational efforts to process unnecessary calculations, significantly limiting the system’s performance. To reduce the unnecessary calculations, researchers propose sparse attention to convert some dense-dense matrices multiplication (DDMM) operations to sampled dense-dense matrix multiplication (SDDMM) and sparse matrix multiplication (SpMM) operations. However, current sparse attention solutions introduce massive off-chip random memory access since the sparse attention matrix is generally unstructured. We propose CPSAA, a novel crossbar-based processing-in-memory (PIM)-featured sparse attention accelerator to eliminate off-chip data transmissions. First, we present a novel attention calculation mode to balance the crossbar writing and crossbar processing latency. Second, we design a novel PIM-based sparsity pruning architecture to eliminate the pruning phase’s off-chip data transfers. Finally, we present novel crossbar-based SDDMM and SpMM methods to process unstructured sparse attention matrices by coupling two types of crossbar arrays. Experimental results show that CPSAA has an average of 89.6×, 32.2×, 17.8×, 3.39×, and 3.84× performance improvement and 755.6×, 55.3×, 21.3×, 5.7×, and 4.9× energy-saving when compare with GPU, FPGA, SANGER, ReBERT, and ReTransformer. Copyright © 2022, The Authors. All rights reserved.

关键词： Memory architecture

来源：评论

学校读者我要写书评

暂无评论

Design and Simulation of Multi-tiered Heterogeneous Memory Architecture

Design and Simulation of Multi-tiered Heterogeneous Memory A...

引用

International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS)

作者： Jinyuan Hu Haikun Liu Hai Jin Xiaofei Liao National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China

Heterogeneous memory systems have become increasingly popular in recent years. Because heterogeneous storage media often show significantly different characteristics in terms of bandwidth, latency, capacity, and energy consumption, it is still challenging to best utilize them for cost-efficient and energy-efficient heterogeneous memory systems. In this paper, we propose a simulation framework for multi-tiered heterogeneous memory architectures based on GEM5 and DRAMsim3 simulators. We design a heterogeneous memory controller to architect Non-Volatile Memory (NVM) as main memory, and architect both Dynamic Random Access Memory (DRAM) and High-Bandwidth Memory (HBM) as a hybrid cache of NVM. Specifically, HBM, DRAM, and NVM are managed in a single (flat) address space. However, we use an address remapping table to maintain the mappings between NVM pages and HBM/DRAM pages, and logically manage HBM/DRAM/NVM as a three-tiered hybrid memory system. We also design a hardware-supported hot page monitor based on Majority Element Algorithm (MEA) to identify the hottest pages in the DRAM, and a dynamic threshold adjustment scheme for hot page migration to balance the memory bandwidth between DRAM and HBM. Our multi-tiered heterogeneous memory architecture can take advantage of the large capacity of NVM, the low latency of DRAM, and the high bandwidth of HBM concurrently. Experimental results show that our tiered memory architecture can improve application performance by an average of $2.5\times$ compared with an NVM-only architecture, and up to 57.4% compared with a DRAM-only architecture. Moreover, the performance gap between our HBM/DRAM/NVM architecture and a HBM-only architecture is less than 10%.

关键词： Analytical models Nonvolatile memory Heuristic algorithms Computational modeling Memory architecture Random access memory Computer architecture

来源：评论

学校读者我要写书评

暂无评论

FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

arXiv

引用

arXiv 2024年

作者： He, Jialuo Chen, Wei Zhang, Xiaojin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China School of Microelectronics and Communication Engineering Chongqing University Chongqing400044 China School of Software Engineering Huazhong University of Science and Technology Wuhan430074 China

Federated Learning (FL) has emerged as a promising approach for privacy-preserving model training across decentralized devices. However, it faces challenges such as statistical heterogeneity and susceptibility to adversarial attacks, which can impact model robustness and fairness. Personalized FL attempts to provide some relief by customizing models for individual clients. However, it falls short in addressing server-side aggregation vulnerabilities. We introduce a novel method called FedAA, which optimizes client contributions via Adaptive Aggregation to enhance model robustness against malicious clients and ensure fairness across participants in non-identically distributed settings. To achieve this goal, we propose an approach involving a Deep Deterministic Policy Gradient-based algorithm for continuous control of aggregation weights, an innovative client selection method based on model parameter distances, and a reward mechanism guided by validation set performance. Empirically, extensive experiments demonstrate that, in terms of robustness, FedAA outperforms the state-of-the-art methods, while maintaining comparable levels of fairness, offering a promising solution to build resilient and fair federated systems. Our code is available at https://***/Gp1g/FedAA. Copyright © 2024, The Authors. All rights reserved.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

Variational Bayes for Federated Continual Learning

arXiv

引用

arXiv 2024年

作者： Yao, Dezhong Li, Sanmu Dai, Yutong Xu, Zhiqiang Hu, Shengshan Zhao, Peilin Sun, Lichao The National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab. Cluster and Grid Computing Lab. School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China Lehigh University PA18015 United States Mohamed bin Zayed University of Artificial Intelligence United Arab Emirates Tencent AI Lab. Shenzhen China

Federated continual learning (FCL) has received increasing attention due to its potential in handling real-world streaming data, characterized by evolving data distributions and varying client classes over time. The constraints of storage limitations and privacy concerns confine local models to exclusively access the present data within each learning cycle. Consequently, this restriction induces performance degradation in model training on previous data, termed "catastrophic forgetting". However, existing FCL approaches need to identify or know changes in data distribution, which is difficult in the real world. To release these limitations, this paper directs attention to a broader continuous framework. Within this framework, we introduce Federated Bayesian Neural Network (FedBNN), a versatile and efficacious framework employing a variational Bayesian neural network across all clients. Our method continually integrates knowledge from local and historical data distributions into a single model, adeptly learning from new data distributions while retaining performance on historical distributions. We rigorously evaluate FedBNN’s performance against prevalent methods in federated learning and continual learning using various metrics. Experimental analyses across diverse datasets demonstrate that FedBNN achieves state-of-the-art results in mitigating forgetting. Copyright © 2024, The Authors. All rights reserved.

关键词： Digital storage

来源：评论

学校读者我要写书评

暂无评论

Fast and Scalable Gate-Level Simulation in Massively Parallel Systems

Fast and Scalable Gate-Level Simulation in Massively Paralle...

引用

IEEE International Conference on Computer-Aided Design

作者： Haichuan Hu Zichen Xu Yuhao Wang Fangming Liu Services Computing Technology and System Lab Cluster and Grid Computing Lab National Engineering Research Center for Big Data Technology and System School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China School of Mathematics and Computer Science Nanchang University Nanchang China Pengcheng Laboratory Shenzhen China

The natural bijection between a proposed circuit design and its graph representation shall allow any graph optimization algorithm deploying into many-core systems efficiently. However, this process suffers from the exponentially growing overhead and heavy memory footprint with the signal propagation. To conquer the unique challenge, we systematically study the simulation with millions of gates, and identify that the processing complexity could grow exponentially from the signal inputs, the skewness of the computational graph stays. Thus, we present ZhouBi, a fast and scalable gate-level simulation framework to fully exploit the parallelism from many-core systems. ZhouBi contributes in threefolds, (I) a graph representation that colors gate-level netlists and identifies skew partitions based on the graph skewness; (II) A set of heuristic algorithms that picks opportunistic and conservative algorithms to accelerate the simulation; (III) A system facility that supports selective mapping between simulation and many-core, providing a tradeoff between the risk of concurrent simulation fail and performance gain. We have prototyped ZhouBi and evaluated it with practical baselines. ZhouBi can achieve a 27.6× performance gain, as compared to the state-of-the-practice Veriwell without compromising any correctness. Our framework supports large graphs enabling scale-out gate-level simulations for chip design.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Hyperbit: A Financial Temporal Knowledge Graph Data Storage System

SSRN

引用

SSRN 2022年

作者： Yuan, Pingpeng Han, Sheng Zang, Shaoqi Shi, Xuanhua Jin, Hai National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

In the field of finance, data has the characteristics of temporal multi-frequency and heterogeneous high dimension, and the traditional knowledge graph is not suitable to express the temporal relation of financial data, so the financial temporal knowledge graph has become a research hotspot recently. There are a large number of temporal edges in the financial temporal knowledge graph. The use of general graph database management system to store financial knowledge graph data has problems such as large storage scale and slow update speed, and it cannot realize the traceability of financial data, so it cannot meet the needs of the financial field. In order to solve these problems, HyperBit, a storage system for efficient storage and management of financial data, is designed and implemented. Firstly, a temporal quad storage structure is proposed. Secondly, the Log Structured Cumulative Update (LSCU) is proposed to update financial temporal data efficiently. The update log module is designed in the system. When there is data to update, it is not directly updated in the data storage of the system, but recorded in the update log, and the system updates the content of the update log cumulatively. The system manages update logs by partition and chunk, which makes the log storage compact and independent. Besides, it is convenient for parallel processing and it improves the data update speed. Thirdly, a Bidirectional Backtracking by Final and Optimal Root (BBFOR) method is proposed to meet the requirements of efficient traceability of financial data. Unlike traditional version tracing method, BBFOR adopt both the snapshot and incremental methods which balance the space and time cost of version switching, and BBROF consider the version of the access frequency characteristic and set up the final root and the optimal root. Besides, BBROF adopt bidirectional backtracking technology to improve the efficiency of the version switching. Experimental results show that the storage cost

关键词： Switching

来源：评论

学校读者我要写书评

暂无评论

PSMiner: A Pattern-Aware Accelerator for High-Performance Streaming Graph Pattern Mining

PSMiner: A Pattern-Aware Accelerator for High-Performance St...

引用

Design Automation Conference

作者： Hao Qi Yu Zhang Ligang He Kang Luo Jun Huang Haoyu Lu Jin Zhao Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China Zhejiang Lab Zhejiang-HUST Joint Research Center for Graph Processing China Department of Computer Science University of Warwick United Kingdom

Streaming Graph Pattern Mining (GPM) has been widely used in many application fields. However, the existing streaming GPM solution suffers from many unnecessary explorations and isomorphism tests, while the existing static GPM ones require many repetitive operations to compute the full graph. In this paper, we propose a pattern-aware incremental execution approach and design the first streaming GPM accelerator called PSMiner, which integrates multiple optimizations to reduce redundant computation and improve computing efficiency. We have conducted extensive experiments. The results show that compared with the state-of-the-art software and hardware solutions, PSMiner achieves the average speedups of 770.9× and 60.4×, respectively.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：