检索结果-内蒙古大学图书馆

arXiv 2025年

作者： Li, Kewei Kong, Yanwen Xu, Yiping Su, Jianlin Huang, Lan Zhang, Ruochi Zhou, Fengfeng College of Computer Science and Technology Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Jilin Changchun130012 China Moonshot AI Ltd. Beijing100086 China

Since the emergence of research on improving the length extrapolation capabilities of large models in 2021, some studies have made modifications to the scaling factor in the scaled dot-product attention mechanism as part of their proposed methods without rigorous theoretical justifications. To fill this gap, we propose two new scaled temperatures based on entropy invariance to enhance length extrapolation. First, a training-free method InfoScale is designed for dot-product attention, and preserves focus on original tokens during length extrapolation by ensuring consistent entropy. Second, we theoretically analyze the impact of scaling (CosScale) on cosine attention. Experimental data demonstrates that combining InfoScale and CosScale achieves state-of-the-art performance on the GAU-α model with a context window extended to 64 times the training length, and outperforms seven existing methods. Our analysis reveals that significantly increasing CosScale approximates the Windowed Attention, and highlights the significance of attention score dilution as a key challenge in long-range context handling. The code and data are available at https://***/HT-NEKO/InfoScale. © 2025, CC BY-NC-ND.

关键词： Data handling

来源：评论

学校读者我要写书评

暂无评论

Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables

Learning Semantic Textual Similarity via Topic-informed Disc...

引用

2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022

作者： Yu, Erxin Du, Lan Jin, Yuan Wei, Zhepei Chang, Yi School of Artificial Intelligence Jilin University China Key Laboratory of Symbolic Computation and Knowledge Engineering Jilin University China International Center of Future Science Jilin University China Faculty of Information Technology Monash University Australia

Recently, discrete latent variable models have received a surge of interest in both Natural Language Processing (NLP) and Computer Vision (CV), attributed to their comparable performance to the continuous counterparts in representation learning, while being more interpretable in their predictions. In this paper, we develop a topic-informed discrete latent variable model for semantic textual similarity, which learns a shared latent space for sentence-pair representation via vector quantization. Compared with previous models limited to local semantic contexts, our model can explore richer semantic information via topic modeling. We further boost the performance of semantic similarity by injecting the quantized representation into a transformer-based language model with a well-designed semantic-driven attention mechanism. We demonstrate, through extensive experiments across various English language datasets, that our model is able to surpass several strong neural baselines in semantic textual similarity tasks. © 2022 Association for computational Linguistics.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought

arXiv

引用

arXiv 2024年

作者： Huang, Sili Hu, Jifeng Chen, Hechang Sun, Lichao Yang, Bo School of Artificial Intelligence Jilin University China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University China Lehigh University BethlehemPA United States

In-context learning is a promising approach for offline reinforcement learning (RL) to handle online tasks, which can be achieved by providing task prompts. Recent works demonstrated that in-context RL could emerge with self-improvement in a trial-and-error manner when treating RL tasks as an across-episodic sequential prediction problem. Despite the self-improvement not requiring gradient updates, current works still suffer from high computational costs when the across-episodic sequence increases with task horizons. To this end, we propose an In-context Decision Transformer (IDT) to achieve self-improvement in a high-level trial-and-error manner. Specifically, IDT is inspired by the efficient hierarchical structure of human decision-making and thus reconstructs the sequence to consist of high-level decisions instead of low-level actions that interact with environments. As one high-level decision can guide multi-step low-level actions, IDT naturally avoids excessively long sequences and solves online tasks more efficiently. Experimental results show that IDT achieves state-of-the-art in long-horizon tasks over current in-context RL methods. In particular, the online evaluation time of our IDT is 36× times faster than baselines in the D4RL benchmark and 27× times faster in the Grid World benchmark. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Few-Shot Fine-Grained Classification of Histological Images

Few-Shot Fine-Grained Classification of Histological Images

引用

Medical Artificial Intelligence (MedAI), IEEE International Conference on

作者： Yingdong Jiang Jing Huang Zihe Jin Leqi Shen Ziyi Zhang Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education College of Software Jilin University Jilin University ChangChun China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education College of Computer Science and Technology Jilin University Jilin University ChangChun China College of Computer Science and Technology Jilin University ChangChun China

Histological image classification plays a crucial role in cancer diagnosis. However, the acquisition of well-labeled histological images is prohibitively expensive, and obtaining rare abnormal samples is challenging. Therefore, applying few-shot learning methods to histological image classification tasks holds significant clinical value. Nevertheless, existing research predom-inantly relies on coarse-grained image classification approaches based on natural image datasets, which struggle to address the fine-grained challenges encountered in histological image classification, such as intra-class diversity and inter-class similarity. To tackle this issue, this study proposes a novel few-shot fine-grained classification method for histological images, named “Category-Aware Feature Map Reconstruction Network.” This method employs channel weights to localize the differences between inter-class and intra-class regions, composed of intra-class channel weights and inter-class channel weights, collectively referred to as category-aware weights. Specifically, intra-class channel weights indicate the matching degree of salient regions within the support set of a particular class, while inter-class channel weights represent the degree of containing distinct information between classes. The category-aware weights are utilized to transform the support feature maps and query feature maps, generating feature maps that capture differentiating details between categories. Finally, the distance between the transformed query feature map and support feature map is calculated to achieve probabilistic predictions for the categories. On a histological few-shot dataset, this method achieves an accuracy of 90.23% using ResNet-12 as the feature extractor, surpassing the baseline model by 5.24% and outperforming other few-shot methods by at least 10% in the 5-way 10-shot experimental setting. The proposed method exhibits exceptional performance on histological image few-shot datasets, playing a

关键词：

来源：评论

学校读者我要写书评

暂无评论

Enhancing Nested Named Entity Recognition in Chinese EMRs: A Collaborative Approach

Enhancing Nested Named Entity Recognition in Chinese EMRs: A...

引用

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Hongtao Yue Xiaoyue Feng Hanchi Xu Haoxuan Chu Yushi Wang Renchu Guan Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education College of Computer Science and Technology Jilin University Changchun China Cardiovascular Center The First Hospital of Jilin University Changchun China

ISBN: (数字)9798350386226

ISBN: (纸本)9798350386233

Named entity recognition (NER) in electronic medical records (EMRs) is critical for identifying medical entities, constructing medical knowledge graphs, and supporting clinical decision-making. However, the scarcity of EMR datasets and the challenges posed by the complexity of Chinese medical texts hinder progress in this area. To address these issues, we introduce CMR-NER, a nested named entity recognition model that enhances entity prediction by integrating multiple features and considering a global view of entity boundaries. Leveraging the robust generalization capabilities of the large language model, CMR-NER involves collaboration with medical experts to ensure accuracy and reduce annotation costs. Additionally, we present HF-NER, a high-quality dataset specifically curated for Chinese EMR entity recognition focusing on heart failure. This dataset, constructed through a methodology combining ChatGPT’s capabilities and expert validation, is a significant contribution to the field. It facilitates a comprehensive evaluation of CMR-NER. Experimental results demonstrate that our approach achieves comparable or superior performance to existing methods, underscoring its effectiveness in this challenging domain.

关键词： Costs Annotations Large language models Collaboration Named entity recognition Medical services knowledge graphs Predictive models Chatbots Cardiovascular diseases

来源：评论

学校读者我要写书评

暂无评论

LLM-Powered User Simulator for Recommender System

arXiv

引用

arXiv 2024年

作者： Zhang, Zijian Liu, Shuchang Liu, Ziru Zhong, Rui Cai, Qingpeng Zhao, Xiangyu Zhang, Chunxu Liu, Qidong Jiang, Peng Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University China Kuaishou Technology China City University of Hong Kong Hong Kong Xi’an Jiaotong University China

User simulators can rapidly generate a large volume of timely user behavior data, providing a testing platform for reinforcement learning-based recommender systems, thus accelerating their iteration and optimization. However, prevalent user simulators generally suffer from significant limitations, including the opacity of user preference modeling and the incapability of evaluating simulation accuracy. In this paper, we introduce an LLM-powered user simulator to simulate user engagement with items in an explicit manner, thereby enhancing the efficiency and effectiveness of reinforcement learning-based recommender systems training. Specifically, we identify the explicit logic of user preferences, leverage LLMs to analyze item characteristics and distill user sentiments, and design a logical model to imitate real human engagement. By integrating a statistical model, we further enhance the reliability of the simulation, proposing an ensemble model that synergizes logical and statistical insights for user interaction simulations. Capitalizing on the extensive knowledge and semantic generation capabilities of LLMs, our user simulator faithfully emulates user behaviors and preferences, yielding high-fidelity training data that enrich the training of recommendation algorithms. We establish quantifying and qualifying experiments on five datasets to validate the simulator’s effectiveness and stability across various recommendation scenarios. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

Boosting Short Text Classification with Multi-Source Information Exploration and Dual-Level Contrastive Learning

arXiv

引用

arXiv 2025年

作者： Liu, Yonghao Li, Mengyu Pang, Wei Giunchiglia, Fausto Huang, Lan Feng, Xiaoyue Guan, Renchu Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education College of Computer Science and Technology Jilin University China Mathematical and Computer Sciences Heriot-Watt University United Kingdom University of Trento Italy

Short text classification, as a research subtopic in natural language processing, is more challenging due to its semantic sparsity and insufficient labeled samples in practical scenarios. We propose a novel model named MI-DELIGHT for short text classification in this work. Specifically, it first performs multi-source information (i.e., statistical information, linguistic information, and factual information) exploration to alleviate the sparsity issues. Then, the graph learning approach is adopted to learn the representation of short texts, which are presented in graph forms. Moreover, we introduce a dual-level (i.e., instance-level and cluster-level) contrastive learning auxiliary task to effectively capture different-grained contrastive information within massive unlabeled data. Meanwhile, previous models merely perform the main task and auxiliary tasks in parallel, without considering the relationship among tasks. Therefore, we introduce a hierarchical architecture to explicitly model the correlations between tasks. We conduct extensive experiments across various benchmark datasets, demonstrating that MI-DELIGHT significantly surpasses previous competitive models. It even outperforms popular large language models on several datasets. Copyright © 2025, The Authors. All rights reserved.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

Testing Non-Commutativity of Reduce Functions with Multi-Column Inputs

SSRN

引用

SSRN 2024年

作者： Zhang, Xuan Zhu, Chenlu Li, Ning Zhang, Peng Liu, Lei College of Software Jilin University Jilin China College of Computer Science and Technology Jilin University Jilin China Key Laboratory of Symbolic Computation and Knowledge Engineering Jilin University Jilin China

With the continuous development of the MapReduce programming model, it is necessary to ensure the reliability of MapReduce programs. In practice, the non-commutativity of Reduce functions seriously affects the reliability of the MapReduce program, which is difficult to debug and even causes errors. Current researches on the non-commutability detection of Reduce function consider the case that the input value is a single attribute. However, such researches ignore the situation where inputs to most reduce functions in practical applications consist of multiple columns (such as a table). To test the commutativity of reduce functions where each input record may contain several input attributes, a new testing method is proposed. This approach uses symbolic execution tools to help generate a few input records, and breaks their data dependencies to generate an original test case t0, with a dynamic program slicing technique to lessen the scale of t0. And the ultimate test suite is consisted of different permutations of records in t0. In the end, experiments demonstrate the effectiveness of our testing method and that the permutation method Gm is helpful to reduce its complexity. © 2024, The Authors. All rights reserved.

关键词： MapReduce

来源：评论

学校读者我要写书评

暂无评论

VECLLF: A vehicle-edge collaborative lifelong learning framework for anomaly detection in VANETs

引用

Computer Networks 2025年 265卷

作者： Wang, Yingqing Liang, Yanhua Huang, Yue Qin, Guihe Jilin University College of Computer Science and Technology Changchun130012 China Jilin University Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Changchun130012 China Jilin University Center for Computer Fundamental Education Changchun130012 China

With the rapid development of intelligent connected vehicles, Vehicular Ad Hoc Network (VANET) is widely used in intelligent transportation systems. VANETs are vulnerable to attacks in their operating environments, such as denial of service (DoS) attacks and injection attacks. Thus, the development of an efficient anomaly detection framework is a promising solution. Most existing methods can only detect known attacks, are less efficient in detecting unknown attacks, and fail to deal with them. To solve these problems, the lifelong learning framework (LLF) is introduced, which aims to store and label suspicious data by applying blockchain and then retrain the model. However, existing LLFs are overly idealized as they do not adequately consider the key issues of label budget and storage space. To improve the practicality of existing methods, this paper proposes a novel LLF based on the existing LLF, called Vehicle Edge Collaborative Lifelong Learning Framework (VECLLF), which can deploy the model update task on the edge server. Specifically, the proposed anomaly detection module based on ensemble active learning is used to achieve high-performance detection under a limited label budget, the proposed incremental learning module based on a sample library continuously learns the model by replaying representative samples within limited storage space, the proposed optimization-based feature selection module is used to remove redundant and interfering features, and the explainable artificial intelligence (XAI) module is used to explain the model to ensure its interpretability. In addition, the blockchain-based database (BC-DB) module is used to help store suspicious data and interact with security agencies to label it. We conducted experiments on two public datasets, AWID2 and CICIDS2017, and the results show that VECLLF outperforms state-of-the-art anomaly detection models and existing LLF, with F1-scores exceeding 97% on both datasets. Furthermore, we collect data from th

关键词： Rapid transit

来源：评论

学校读者我要写书评

暂无评论

A Morphology Focused Cell Detection Model for Histopathology Images

A Morphology Focused Cell Detection Model for Histopathology...

引用

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Zhe Wang Fangyue Wei Shuyu Guo Xiaoting Che Tian Bai College of Computer Science and Technology Jilin University Changchun China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun China

The accurate automatic recognition of cell locations is of great significance for downstream tasks in pathology. Due to the various size and distribution of different cell types, previous cell detection methods applied fixed circles as labels to localize cell position by default, which makes less precise and more difficult to adapt various cell morphology. In this paper, we integrate adaptive areas of interests based on morphology to address the limitation in detecting cells with irregular shapes in pathological images. Moreover, a EGSI module is proposed to extract global distribution of cells in slices, which enables model to acquire more rich semantic information. We exhibit favorable F 1 score of our method on four public datasets. Experimental results demonstrate that the proposed method could detect cells more accurately in those images with different cell morphologies and dense cell distribution.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：