检索结果-内蒙古大学图书馆

KAN v.s. MLP for Offline Reinforcement Learning

学校读者我要写书评

暂无评论

KAN v.s. MLP for Offline Reinforcement Learning

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Guo, Haihong Li, Fengxin Li, Jiao Liu, Hongyan School of Information Renmin University of China China Institute of Medical Information Medical Library Chinese Academy of Medical Sciences Peking Union Medical College China Key Laboratory of Data Engineering and Knowledge Engineering Ministry of Education China School of Economics and Management Tsinghua University China

ISBN: (纸本)9798350368741

Kolmogorov-Arnold Networks (KAN) is an emerging neural network architecture in machine learning. It has greatly interested the research community about whether KAN can be a promising alternative to the commonly used Multi-Layer Perceptions (MLP). Experiments in various fields demonstrated that KAN-based machine learning can achieve comparable if not better performance than MLP-based methods, but with much smaller parameter scales and are more explainable. In this paper, we explore the incorporation of KAN into the actor and critic networks for offline reinforcement learning (RL). We evaluated the performance, parameter scales, and training efficiency of various KAN and MLP-based conservative Q-learning (CQL) on the classical D4RL benchmark for offline RL. Our study demonstrates that KAN can achieve performance close to the commonly used MLP with significantly fewer parameters. This allows us to choose the base networks according to the offline RL task requirements. © 2025 IEEE.

关键词： KAN Kolmogorov-Arnold networks MLP multilayer perceptrons offline reinforcement learning

Dual-Aspect Noise-Based Regularization for Multi-Modal Relation Extraction in Media Posts

学校读者我要写书评

暂无评论

IEEE Transactions on Audio, Speech and Language Processing

IEEE Transactions on Audio, Speech and Language Processing 2025年 33卷 1324-1336页

作者： Kai Sun Bin Shi Samuel Mensah Wenjian Liu Bo Dong School of Computer Science and Technology and Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Xi'an Jiaotong University Xi'an China School of Computer Science and Technology The University of Sheffield Sheffield U.K. Faculty of Data Science City University of Macau Macau China School of Continuing Education and Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Xi'an Jiaotong University Xi'an China

Multi-Modal Relation Extraction (MMRE) plays a key role in various multimedia applications including, recommendation and information retrieval systems. MMRE aims to extract the semantic relation between entities by leveraging context from a text-image pair. By utilizing context from images, the challenge of learning from noisy images in MMRE emerges as a research problem by itself. For instance, subtle variations in similar images can act as noise and potentially impact the predictions made by MMRE models. To tackle this problem, current work utilizes attention mechanisms to fuse relevant text and image features or devise data augmentation techniques (e.g., via generative models) to improve generalization. However, the current performance still remains unsatisfactory. In an effort to improve upon the performance, we propose a Dual-Aspect Noise-based Regularization framework that encompasses two techniques: 1) noise removal through an adaptive gating mechanism, 2) fighting noise with noise to improve feature stability in the learning process. We find that combining these techniques encourages the model to focus on more relevant image features for MMRE. We carry out extensive experiments and demonstrate that our proposed model is further enhanced by exploring data augmentation techniques. This additional improvement leads the model to achieve state-of-the-art performance on the widely-used Multi-modal Neural Relation Extraction (MNRE) dataset, and show its effectiveness and generalizability on the Multi-Modal Named Entity Recognition task.

关键词： Feature extraction Noise data mining Noise measurement Social networking (online) Adaptation models Transformers Training Predictive models data models

Fair Personalized Learner Modeling Without Sensitive Attributes 25

学校读者我要写书评

暂无评论

Fair Personalized Learner Modeling Without Sensitive Attribu...

Proceedings of the ACM on Web Conference 2025

作者： Hefei Xu Min Hou Le Wu Fei Liu Yonghui Yang Haoyue Bai Richang Hong Meng Wang Key Laboratory of Knowledge Engineering with Big Data Hefei University of Technology Hefei Anhui China Key Laboratory of Knowledge Engineering with Big Data Hefei University of Technology Hefei Anhui China and Institute of Dataspace Hefei Comprehensive National Science Center Hefei Hefei Anhui China

ISBN: (纸本)9798400712746

Personalized learner modeling uses learners' historical behavior data to diagnose their cognitive abilities, a process known as Cognitive Diagnosis (CD). This is essential for web-based learning services such as learning resource recommendation and adaptive testing. However, prior studies have shown that CD models may unfairly correlate learners' abilities with sensitive attributes (e.g., gender, region), leading to biased outcomes. While existing approaches mitigate this issue by decorrelating sensitive attributes from the modeling process, privacy concerns make collecting such attributes impractical. Furthermore, the presence of multiple sensitive attributes complicates fairness improvements. In this paper, we explore how to achieve fair personalized learner modeling without relying on any sensitive attribute input. We introduce a novel fairness objective tailored for personalized learner modeling and design a max-min strategy to facilitate both sensitive information inference and fair CD modeling. In the max step, we infer pseudo-labels by maximizing the fairness objective, while in the min step, we retrain the CD model by minimizing it. Additionally, we provide a theoretical guarantee that our framework reduces the upper bound of fairness generalization error. Extensive experiments demonstrate that the proposed framework significantly outperforms existing methods. Our code is available at: https://***/HeFei-X/FairWISA.

关键词： cognitive diagnosis

A Part-of-Speech Tagging Model Employing Word Clustering and Syntactic Parsing

学校读者我要写书评

暂无评论

Chinese Journal of Electronics 2025年第1期23卷 109-114页

作者： Lichi Yuan School of Information Technology Jiangxi University of Finance and Economics Nanchang China Jiangxi Key Laboratory of Data and Knowledge Engineering Jiangxi University of Finance and Economics Nanchang China

Part-Of-Speech tagging is a basic task in the field of natural language processing. This paper builds a POS tagger based on improved Hidden Markov model, by employing word clustering and syntactic parsing model. Firstly, In order to overcome the defects of the classical HMM, Markov family model (MFM), a new statistical model was introduced. Secondly, to solve the problem of data sparseness, we propose a bottom-to-up hierarchical word clustering algorithm. Then we combine syntactic parsing with part-of-speech tagging. The Part-of-Speech tagging experiments show that the improved Part-Of-Speech tagging model has higher performance than Hidden Markov models (HMMs) under the same testing conditions, the precision is enhanced from 94.642% to 97.235%.

关键词：

A Query Optimization Method Utilizing Large Language Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yao, Zhiming Li, Haoyang Zhang, Jing Li, Cuiping Chen, Hong School of Information Renmin University of China Beijing China Key Laboratory of Data Engineering and Knowledge Engineering MOE China Engineering Research Center of Database and Business Intelligence MOE China

Query optimization is a critical task in database systems, focused on determining the most efficient way to execute a query from an enormous set of possible strategies. Traditional approaches rely on heuristic search methods and cost predictions, but these often struggle with the complexity of the search space and inaccuracies in performance estimation, leading to suboptimal plan choices. This paper presents LLMOpt, a novel framework that leverages Large Language Models (LLMs) to address these challenges through two innovative components: (1) LLM for Plan Candidate Generation (LLMOpt(G)), which eliminates heuristic search by utilizing the reasoning abilities of LLMs to directly generate high-quality query plans, and (2) LLM for Plan Candidate Selection (LLMOpt(S)), a list-wise cost model that compares candidates globally to enhance selection accuracy. To adapt LLMs for query optimization, we propose fine-tuning pre-trained models using optimization data collected offline. Experimental results on the JOB, JOB-EXT, and Stack benchmarks show that LLMOpt(G) and LLMOpt(S) outperform state-of-the-art methods, including PostgreSQL, BAO, and HybridQO. Notably, LLMOpt(S) achieves the best practical performance, striking a balance between plan quality and inference efficiency. Copyright © 2025, The Authors. All rights reserved.

关键词： Structured Query Language

LLMIdxAdvis: Resource-Efficient Index Advisor Utilizing Large Language Model

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhao, Xinxin Li, Haoyang Zhang, Jing Huang, Xinmei Zhang, Tieying Chen, Jianjun Shi, Rui Li, Cuiping Chen, Hong School of Information Renmin University of China Beijing China Key Laboratory of Data Engineering and Knowledge Engineering MOE China Engineering Research Center of Database and Business Intelligence MOE China ByteDance China

Index recommendation is essential for improving query performance in database management systems (DBMSs) through creating an optimal set of indexes under specific constraints. Traditional methods, such as heuristic and learning-based approaches, are effective but face challenges like lengthy recommendation time, resource-intensive training, and poor generalization across different workloads and database schemas. To address these issues, we propose LLMIdxAdvis, a resource-efficient index advisor that uses large language models (LLMs) without extensive fine-tuning. LLMIdxAdvis frames index recommendation as a sequence-to-sequence task, taking target workload, storage constraint, and corresponding database environment as input, and directly outputting recommended indexes. It constructs a high-quality demonstration pool offline, using GPT-4-Turbo to synthesize diverse SQL queries and applying integrated heuristic methods to collect both default and refined labels. During recommendation, these demonstrations are ranked to inject database expertise via in-context learning. Additionally, LLMIdxAdvis extracts workload features involving specific column statistical information to strengthen LLM’s understanding, and introduces a novel inference scaling strategy combining vertical scaling (via "Index-Guided Major Voting" and Best-of-N) and horizontal scaling (through iterative "self-optimization" with database feedback) to enhance reliability. Experiments on 3 OLAP and 2 real-world benchmarks reveal that LLMIdxAdvis delivers competitive index recommendation with reduced runtime, and generalizes effectively across different workloads and database schemas. Copyright © 2025, The Authors. All rights reserved.

关键词： Query languages

How to Mitigate Information Loss in knowledge Graphs for GraphRAG: Leveraging Triple Context Restoration and Query-Driven Feedback

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Huang, Manzong Bu, Chenyang He, Yi Wu, Xindong Key Laboratory of Knowledge Engineering with Big Data [Hefei University of Technology Ministry of Education China School of Data Science William & Mary WilliamsburgVA United States

knowledge Graph (KG)-augmented Large Language Models (LLMs) have recently propelled significant advances in complex reasoning tasks, thanks to their broad domain knowledge and contextual awareness. Unfortunately, current methods often assume KGs to be complete, which is impractical given the inherent limitations of KG construction and the potential loss of contextual cues when converting unstructured text into entity-relation triples. In response, this paper proposes the Triple Context Restoration and Query-driven Feedback (TCR-QF) framework, which reconstructs the textual context underlying each triple to mitigate information loss, while dynamically refining the KG structure by iteratively incorporating query-relevant missing knowledge. Experiments on five benchmark question-answering datasets substantiate the effectiveness of TCR-QF in KG and LLM integration, where it achieves a 29.1% improvement in Exact Match and a 15.5% improvement in F1 over its state-of-the-art GraphRAG competitors. © 2025, CC BY.

关键词： knowledge graph

OmniSQL: Synthesizing High-quality Text-to-SQL data at Scale

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Li, Haoyang Wu, Shang Zhang, Xiaokang Huang, Xinmei Zhang, Jing Jiang, Fuxin Wang, Shuai Zhang, Tieying Chen, Jianjun Shi, Rui Chen, Hong Li, Cuiping Engineering Research Center of Database and Business Intelligence MOE China School of Information Renmin University of China Beijing China Key Laboratory of Data Engineering and Knowledge Engineering MOE China ByteDance Inc China

Text-to-SQL, the task of translating natural language questions into SQL queries, plays a crucial role in enabling non-experts to interact with databases. While recent advancements in large language models (LLMs) have significantly enhanced text-to-SQL performance, existing approaches face notable limitations in real-world text-to-SQL applications. Prompting-based methods often depend on closed-source LLMs, which are expensive, raise privacy concerns, and lack customization. Fine-tuning-based methods, on the other hand, suffer from poor generalizability due to the limited coverage of publicly available training data. To overcome these challenges, we propose a novel and scalable text-to-SQL data synthesis framework for automatically synthesizing large-scale, high-quality, and diverse datasets without extensive human intervention. Using this framework, we introduce SynSQL-2.5M, the first million-scale text-to-SQL dataset, containing 2.5 million samples spanning over 16,000 synthetic databases. Each sample includes a database, SQL query, natural language question, and chain-of-thought (CoT) solution. Leveraging SynSQL-2.5M, we develop OmniSQL, a powerful open-source text-to-SQL model available in three sizes: 7B, 14B, and 32B. Extensive evaluations across nine datasets demonstrate that OmniSQL achieves state-of-the-art performance, matching or surpassing leading closed-source and open-source LLMs, including GPT-4o and DeepSeek-V3, despite its smaller size. We release all code, datasets, and models to support further research. Copyright © 2025, The Authors. All rights reserved.

关键词： Structured Query Language

Towards a Theoretical Understanding of Semi-Supervised Learning Under Class Distribution Mismatch

学校读者我要写书评

暂无评论

IEEE Transactions on Pattern Analysis and Machine Intelligence 2025年第6期47卷 4853-4868页

作者： Pan Du Suyun Zhao Puhui Tan Zisen Sheng Zeyu Gan Hong Chen Cuiping Li School of Information Renmin University of China Beijing China Engineering Research Center of Database and Business Intelligence Ministry of Education (MOE) Beijing China Key Laboratory of Data Engineering and Knowledge Engineering Ministry of Education (MOE) Beijing China School of Statistics Renmin University of China Beijing China

Semi-supervised learning (SSL) confronts a formidable challenge under class distribution mismatch, wherein unlabeled data contain numerous categories absent in the labeled dataset. Traditional SSL methods undergo performance deterioration in such mismatch scenarios due to the invasion of those instances from unknown categories. Despite some technical efforts to enhance SSL by mitigating the invasion, the profound theoretical analysis of SSL under class distribution mismatch is still under study. Accordingly, in this work, we propose Bi- Objective Optimization Mechanism (BOOM) to theoretically analyze the excess risk between the empirical optimal solution and the population-level optimal solution. Specifically, BOOM reveals that the SSL error is the essential contributor behind excess risk, resulting from both the pseudo-labeling error and invasion error. Meanwhile, BOOM unveils that the optimization objectives of SSL under mismatch are binary: high-quality pseudo-labels and adaptive weights on the unlabeled instances, which contribute to alleviating the pseudo-labeling error and the invasion error, respectively. Moreover, BOOM explicitly discovers the fundamental factors crucial for optimizing the bi-objectives, guided by which an approach is then proposed as a strong baseline for SSL under mismatch. Extensive experiments on benchmark and real datasets confirm the effectiveness of our proposed algorithm.

关键词： Training Labeling Semisupervised learning Predictive models Optimization Perturbation methods Entropy data models Annotations Upper bound