检索结果-内蒙古大学图书馆

Distilling Autoregressive Models to Obtain High-Performance Non-Autoregressive Solvers for Vehicle Routing Problems with Faster Inference Speed

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xiao, Yubin Wang, Di Li, Boyang Wang, Mingzhao Wu, Xuan Zhou, Changliang Zhou, You Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education College of Computer Science and Technology Jilin University China Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly Nanyang Technological University Singapore WeBank-NTU Joint Research Institute on Fintech Nanyang Technological University Singapore School of Computer Science and Engineering Nanyang Technological University Singapore School of System Design and Intelligent Manufacturing Southern University of Science and Technology China

Neural construction models have shown promising performance for Vehicle Routing Problems (VRPs) by adopting either the Autoregressive (AR) or Non-Autoregressive (NAR) learning approach. While AR models produce high-quality solutions, they generally have a high inference latency due to their sequential generation nature. Conversely, NAR models generate solutions in parallel with a low inference latency but generally exhibit inferior performance. In this paper, we propose a generic Guided Non-Autoregressive knowledge Distillation (GNARKD) method to obtain high-performance NAR models having a low inference latency. GNARKD removes the constraint of sequential generation in AR models while preserving the learned pivotal components in the network architecture to obtain the corresponding NAR models through knowledge distillation. We evaluate GNARKD by applying it to three widely adopted AR models to obtain NAR VRP solvers for both synthesized and real-world instances. The experimental results demonstrate that GNARKD significantly reduces the inference time (4∼5× faster) with acceptable performance drop (2∼3%). To the best of our knowledge, this study is first-of-its-kind to obtain NAR VRP solvers from AR ones through knowledge distillation. Copyright © 2023, The Authors. All rights reserved.

关键词： Distillation

Multimodal Hypothetical Summary for Retrieval-based Multi-image Question Answering

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Peize Si, Qingyi Fu, Peng Lin, Zheng Wang, Yan School of Artificial Intelligence Jilin University Changchun China Institute of Information Engineering Chinese Academy of Sciences Beijing China Huawei Technologies Co. Ltd. Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Key Laboratory of Symbol Computation and Knowledge Engineering Ministry of Education College of Computer Science and Technology Jilin University Changchun China

Retrieval-based multi-image question answering (QA) task involves retrieving multiple question-related images and synthesizing these images to generate an answer. Conventional "retrieve-then-answer" pipelines often suffer from cascading errors because the training objective of QA fails to optimize the retrieval stage. To address this issue, we propose a novel method to effectively introduce and reference retrieved information into the QA. Given the image set to be retrieved, we employ a multimodal large language model (visual perspective) and a large language model (textual perspective) to obtain multimodal hypothetical summary in question-form and description-form. By combining visual and textual perspectives, MHyS captures image content more specifically and replaces real images in retrieval, which eliminates the modality gap by transforming into text-to-text retrieval and helps improve retrieval. To more advantageously introduce retrieval with QA, we employ contrastive learning to align queries (questions) with MHyS. Moreover, we propose a coarse-to-fine strategy for calculating both sentence-level and word-level similarity scores, to further enhance retrieval and filter out irrelevant details. Our approach achieves a 3.7% absolute improvement over state-of-the-art methods on RETVQA and a 14.5% improvement over CLIP. Comprehensive experiments and detailed ablation studies demonstrate the superiority of our method. Copyright © 2024, The Authors. All rights reserved.

关键词： Question answering

AutoSTL: Automated Spatio-Temporal Multi-Task Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Zijian Zhao, Xiangyu Miao, Hao Zhang, Chunxu Zhao, Hongwei Zhang, Junbo College of Computer Science and Technology Jilin University China School of Data Science City University of Hong Kong Hong Kong Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University China Department of Computer Science Aalborg University Denmark JD Intelligent Cities Research China JD iCity JD Technology China Hong Kong Institute for Data Science City University of Hong Kong Hong Kong

Spatio-temporal prediction plays a critical role in smart city construction. Jointly modeling multiple spatio-temporal tasks can further promote an intelligent city life by integrating their inseparable relationship. However, existing studies fail to address this joint learning problem well, which generally solve tasks individually or a fixed task combination. The challenges lie in the tangled relation between different properties, the demand for supporting flexible combinations of tasks and the complex spatio-temporal dependency. To cope with the problems above, we propose an Automated Spatio-Temporal multi-task Learning (AutoSTL) method to handle multiple spatio-temporal tasks jointly. Firstly, we propose a scalable architecture consisting of advanced spatio-temporal operations to exploit the complicated dependency. Shared modules and feature fusion mechanism are incorporated to further capture the intrinsic relationship between tasks. Furthermore, our model automatically allocates the operations and fusion weight. Extensive experiments on benchmark datasets verified that our model achieves state-of-the-art performance. As we can know, AutoSTL is the first automated spatio-temporal multi-task learning method. Copyright © 2023, The Authors. All rights reserved.

关键词： Smart city

ESAE: Evolutionary Strategy-Based Architecture Evolution 14th

学校读者我要写书评

暂无评论

ESAE: Evolutionary Strategy-Based Architecture Evolution

14th International Conference on Bio-inspired Computing: Theories and Applications, BIC-TA 2019

作者： Gu, Xue Meng, Ziyao Liang, Yanchun Xu, Dong Huang, Han Han, Xiaosong Wu, Chunguo Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education College of Computer Science and Technology Jilin University Changchun130012 China Zhuhai Laboratory of Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education School of Computer Zhuhai College of Jilin University Zhuhai519041 China Department of Electrical Engineering and Computer Science Informatics Institute and Christopher S. Bond Life Sciences Center University of Missouri ColumbiaMO65211 United States School of Software Engineering South China University of Technology GuangZhou510006 China

ISBN: (纸本)9789811534249

Although deep neural networks (DNNs) play important roles in many fields, the architecture design of DNNs can be challenging due to the difficulty of input data representation, the huge number of parameters and the complex layer relationships. To overcome the obstacles of architecture design, we developed a new method to generate the optimal structure of DNNs, named Evolutionary Strategy-based Architecture Evolution (ESAE), consisting of a bi-level representation and a probability distribution learning approach. The bi-level representation encodes architectures in the gene and parameter levels. The probability distribution learning approach ensures the efficient convergence of the architecture searching process. By using Fashion-MNIST and CIFAR-10, the effectiveness of the proposed ESAS is verified. The evolved DNNs, starting from a trivial initial architecture with one single convolutional layer, achieved the accuracies of 94.48% and 93.49% on Fashion-MNIST and CIFAR-10, respectively, and require remarkably less hardware costs in terms of GPUs and running time, compared with the existing state-of-the-art manual screwed architectures. © 2020, Springer Nature Singapore Pte Ltd.

关键词： Deep neural networks

Bi-classifier with neighborhood aggregation for unsupervised domain adaptation

学校读者我要写书评

暂无评论

Information Sciences 2025年 718卷

作者： Shuai Lü Xinyu Zhang Zongze Li Jingyao Li Meng Kang Key Laboratory of Symbolic Computation and Knowledge Engineering (Jilin University) Ministry of Education Changchun 130012 China College of Computer Science and Technology Jilin University Changchun 130012 China College of Software Jilin University Changchun 130012 China

Recently, adversarial learning with bi-classifier has been confirmed to be effective in unsupervised domain adaptation, which can effectively promote the alignment of different distributions across domains. Most previous methods with bi-classifier exploit the output disagreement of the bi-classifier to learn transferable representations. However, only considering the disagreement between two classifiers cannot guarantee that the model performs well in the target domain. The main reason is that the model does not consider the local structure of the target domain data. To address this problem, we propose a Bi-Classifier with Neighborhood Aggregation (BCNA). Specifically, we provide two pseudo-labels for a target sample based on the nearest neighbor aggregation strategy. Then, we propose a self-supervised loss with two pseudo-labels based on confidence weight. In addition, we also propose a novel consistency regularization loss for bi-classifier. Unlike previous methods with bi-classifier, we achieve the goal of minimizing the disagreement between two classifiers from the perspective of different views of the sample. Our method can increase the discriminability of the model while ensuring the feature transferability. Extensive experiments show that our method has achieved state-of-the-art performance on three datasets that are widely used in unsupervised domain adaptation.

关键词： Bi-classifier Consistency regularization Neighborhood aggregation Self-supervision Unsupervised domain adaptation

A Survey of Sum-Product Networks Structural Learning

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Xia, Riting Zhang, Yan Liu, Xueyan Yang, Bo School of Artificial Intelligence Jilin University Jilin Changchun130012 China School of Computer Science and Technology Jilin University Jilin Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineer Jilin University Ministry of Education Jilin Changchun130012 China College of Communication Engineering Jilin University Jilin Changchun130012 China

Sum-product networks (SPNs) in deep probabilistic models have made great progress in computer vision, robotics, neuro-symbolic artificial intelligence, natural language processing, probabilistic programming languages, and other fields. Compared with probabilistic graphical models and deep probabilistic models, SPNs can balance the tractability and expressive efficiency. In addition, SPNs remain more interpretable than deep neural models. The expressiveness and complexity of SPNs depend on its own structure. Thus, how to design an effective SPN structure learning algorithm that can balance expressiveness and complexity has become a hot research topic in recent years. In this paper, we review SPN structure learning comprehensively, including the motivation of SPN structure learning, a systematic review of related theories, the proper categorization of different SPN structure learning algorithms, several evaluation approaches and some helpful online resources. Moreover, we discuss some open issues and research directions for SPN structure learning. To our knowledge, this is the first survey to focus specifically on SPN structure learning, and we hope to provide useful references for researchers in related fields. © 2022, The Authors. All rights reserved.

关键词： Efficiency

Spirit distillation: Precise real-time semantic segmentation of road scenes with insufficient data

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wu, Zhiyuan Jiang, Yu Cui, Chupeng Yang, Zongmin Xue, Xinhui Qi, Hong Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun China With College of Computer Science and Technology Jilin University Changchun China College of Computer Science and Technology Jilin University Changchun China

Semantic segmentation of road scenes is one of the key technologies for realizing autonomous driving scene perception, and the effectiveness of deep Convolutional Neural Networks(CNNs) for this task has been demonstrated. Stateof-art CNNs for semantic segmentation suffer from excessive computations as well as large-scale training data requirement. Inspired by the ideas of Fine-tuning-based Transfer Learning (FTT) and feature-based knowledge distillation, we propose a new knowledge distillation method for cross-domain knowledge transference and efficient data-insufficient network training, named Spirit Distillation(SD), which allow the student network to mimic the teacher network to extract general features, so that a compact and accurate student network can be trained for real-time semantic segmentation of road scenes. Then, in order to further alleviate the trouble of insufficient data and improve the robustness of the student, an Enhanced Spirit Distillation (ESD) method is proposed, which commits to exploit a more comprehensive general features extraction capability by considering images from both the target and the proximity domains as input. To our knowledge, this paper is a pioneering work on the application of knowledge distillation to few-shot learning. Persuasive experiments conducted on Cityscapes semantic segmentation with the prior knowledge transferred from COCO2017 and KITTI demonstrate that our methods can train a better student network (mIOU and high-precision accuracy boost by 1.4% and 8.2% respectively, with 78.2% segmentation variance) with only 41.8% FLOPs (see Fig. 1). Copyright © 2021, The Authors. All rights reserved.

关键词： Autonomous vehicles

Ontology Revision based on Pre-trained Language Models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ji, Qiu Zhang, Xiaoping Ye, Yuxin Qi, Guilin Li, Jiaye Li, Site Ren, Jianjie Lu, Songtao School of Modern Posts Institute of Modern Posts Nanjing University of Posts and Telecommunications Nanjing China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China National Data Center ofTraditional Chinese MedicineChina Academy of ChineseMedical Sciences Beijing100700 China College of Computer Science and Technology Jilin University Changchun130012 China School of Computer Science and Engineering Southeast University Nanjing China Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education Nanjing China School of Mathematics Southeast University Nanjing China Chien-Shiung Wu College Southeast University Nanjing China

Ontology revision aims to seamlessly incorporate a new ontology into an existing ontology and plays a crucial role in tasks such as ontology evolution, ontology maintenance, and ontology alignment. Similar to repair single ontologies, resolving logical incoherence in the task of ontology revision is also important and meaningful, because incoherence is a main potential factor to cause inconsistency and reasoning with an inconsistent ontology will obtain meaningless answers. To deal with this problem, various ontology revision approaches have been proposed to define revision operators and design ranking strategies for axioms in an ontology. However, they rarely consider axiom semantics which provides important information to differentiate axioms. In addition, pre-trained models can be utilized to encode axiom semantics, and have been widely applied in many natural language processing tasks and ontology-related ones in recent years. Therefore, in this paper, we study how to apply pre-trained models to revise ontologies. We first define four scoring functions to rank axioms based on a pre-trained model by considering various information from an ontology. Based on the functions, an ontology revision algorithm is then proposed to deal with unsatisfiable concepts at once. To improve efficiency, an adapted revision algorithm is designed to deal with unsatisfiable concepts group by group. We conduct experiments over 19 ontology pairs and compare our algorithms and scoring functions with existing ones. According to the experiments, our algorithms could achieve promising performance. The adapted revision algorithm could improve the efficiency largely, and at most about 90% of the time could be saved for some ontology pairs. Some of our scoring functions like reliableOnt cos could help a revision algorithm obtain better results in many cases, especially for those challenging ontology pairs like OM8. We also provide discussion about the overall experimental results and guidelin

关键词： Ontology

Enhanced Doubly Robust Learning for Debiasing Post-Click Conversion Rate Estimation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Guo, Siyuan Zou, Lixin Liu, Yiding Ye, Wenwen Cheng, Suqi Wang, Shuaiqiang Chen, Hechang Yin, Dawei Chang, Yi School of Artificial Intelligence Jilin University China Baidu Inc. China International Center of Future Science Jilin University China Key Laboratory of Symbolic Computation Knowledge Engineering of Ministry of Education Jilin University China

Post-click conversion, as a strong signal indicating the user preference, is salutary for building recommender systems. However, accurately estimating the post-click conversion rate (CVR) is challenging due to the selection bias, i.e., the observed clicked events usually happen on users’ preferred items. Currently, most existing methods utilize counterfactual learning to debias recommender systems. Among them, the doubly robust (DR) estimator has achieved competitive performance by combining the error imputation based (EIB) estimator and the inverse propensity score (IPS) estimator in a doubly robust way. However, inaccurate error imputation may result in its higher variance than the IPS estimator. Worse still, existing methods typically use simple model-agnostic methods to estimate the imputation error, which are not sufficient to approximate the dynamically changing model-correlated target (i.e., the gradient direction of the prediction model). To solve these problems, we first derive the bias and variance of the DR estimator. Based on it, a more robust doubly robust (MRDR) estimator has been proposed to further reduce its variance while retaining its double robustness. Moreover, we propose a novel double learning approach for the MRDR estimator, which can convert the error imputation into the general CVR estimation. Besides, we empirically verify that the proposed learning scheme can further eliminate the high variance problem of the imputation learning. To evaluate its effectiveness, extensive experiments are conducted on a semi-synthetic dataset and two real-world datasets. The results demonstrate the superiority of the proposed approach over the state-of-the-art methods. The code is available at https://***/guosyjlu/MRDR-DL. Copyright © 2021, The Authors. All rights reserved.

关键词： Recommender systems