检索结果-内蒙古大学图书馆

arXiv 2025年

作者： Ranasinghe, Nisal Senanayake, Damith Halgamuge, Saman AI Optimization and Pattern Recognition Research Group Dept. of Mechanical Eng. University of Melbourne Australia

—The ability to discover meaningful, accurate, and concise mathematical equations that describe datasets is valuable across various domains. Equations offer explicit relationships between variables, enabling deeper insights into underlying data patterns. Most existing equation discovery methods rely on genetic programming, which iteratively searches the equation space but is often slow and prone to overfitting. By representing equations as directed acyclic graphs, we leverage the use of graph neural networks to learn the underlying semantics of equations, and generate new, previously unseen equations. Although graph generative models have been shown to be successful in discovering new types of graphs in many fields, there application in discovering equations remains largely unexplored. In this work, we propose Graph-EQ, a deep graph generative model designed for efficient equation discovery. Graph-EQ uses a conditional variational autoencoder (CVAE) to learn a rich latent representation of the equation space by training it on a large corpus of equations in an unsupervised manner. Instead of directly searching the equation space, we employ Bayesian optimization to efficiently explore this learned latent space. We show that the encoder-decoder architecture of Graph-Eq is able to accurately reconstruct input equations. Moreover, we show that the learned latent representation can be sampled and decoded into valid equations, including new and previously unseen equations in the training data. Finally, we assess Graph-Eq’s ability to discover equations that best fit a dataset by exploring the latent space using Bayesian optimization. Latent space exploration is done on 20 dataset with known ground-truth equations, and Graph-Eq is shown to successfully discover the grountruth equation in the majority of datasets. © 2025, CC BY-NC-ND.

关键词： Genetic programming

来源：评论

学校读者我要写书评

暂无评论

Rethinking Time Series Forecasting with LLMs via Nearest Neighbor Contrastive Learning

arXiv

引用

arXiv 2024年

作者： Bogahawatte, Jayanie Seneviratne, Sachith Perera, Maneesha Halgamuge, Saman AI Optimization and Pattern Recognition Research Group Dept. of Mechanical Eng. University of Melbourne Australia

Adapting Large Language Models (LLMs) that are extensively trained on abundant text data, and customizing the input prompt to enable time series forecasting has received considerable attention. While recent work has shown great potential for adapting the learned prior of LLMs, the formulation of the prompt to finetune LLMs remains challenging as prompt should be aligned with time series data. Additionally, current approaches do not effectively leverage word token embeddings which embody the rich representation space learned by LLMs. This emphasizes the need for a robust approach to formulate the prompt which utilizes the word token embeddings while effectively representing the characteristics of the time series. To address these challenges, we propose NNCL-TLLM: Nearest Neighbor Contrastive Learning for Time series forecasting via LLMs. First, we generate time series compatible text prototypes such that each text prototype represents both word token embeddings in its neighborhood and time series characteristics via end-to-end finetuning. Next, we draw inspiration from Nearest Neighbor Contrastive Learning to formulate the prompt while obtaining the top-k nearest neighbor time series compatible text prototypes. We then fine-tune the layer normalization and positional embeddings of the LLM, keeping the other layers intact, reducing the trainable parameters and decreasing the computational cost. Our comprehensive experiments demonstrate that NNCL-TLLM outperforms in few-shot forecasting while achieving competitive or superior performance over the state-of-the-art methods in long-term and short-term forecasting tasks. © 2024, CC BY.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

GINN-KAN: Interpretability pipelining with applications in Physics Informed Neural Networks

arXiv

引用

arXiv 2024年

作者： Ranasinghe, Nisal Xia, Yu Seneviratne, Sachith Halgamuge, Saman AI Optimization and Pattern Recognition Research Group Dept. of Mechanical Eng. University of Melbourne Australia

Neural networks are powerful function approximators, yet their "black-box" nature often renders them opaque and difficult to interpret. While many post-hoc explanation methods exist, they typically fail to capture the underlying reasoning processes of the networks. A truly interpretable neural network would be trained similarly to conventional models using techniques such as backpropagation, but additionally provide insights into the learned input-output relationships. In this work, we introduce the concept of interpretability pipelineing, to incorporate multiple interpretability techniques to outperform each individual technique. To this end, we first evaluate several architectures that promise such interpretability, with a particular focus on two recent models selected for their potential to incorporate interpretability into standard neural network architectures while still leveraging backpropagation: the Growing Interpretable Neural Network (GINN) and Kolmogorov Arnold Networks (KAN). We analyze the limitations and strengths of each and introduce a novel interpretable neural network GINN-KAN that synthesizes the advantages of both models. When tested on the Feynman symbolic regression benchmark datasets, GINN-KAN outperforms both GINN and KAN. To highlight the capabilities and the generalizability of this approach, we position GINN-KAN as an alternative to conventional black-box networks in Physics-Informed Neural Networks (PINNs), which we propose as a challenging testbed for backpropagation-friendly interpretable neural networks. By adding interpretability to PINNs, we allow far more transparent and trustworthy data-driven solutions to differential equations. We expect this to have far-reaching implications in the application of deep learning pipelines in the natural sciences. Our experiments with this interpretable PINN on 15 different partial differential equations demonstrate that GINN-KAN augmented PINNs outperform PINNs with black-box networks in solving d

关键词： Regression analysis

来源：评论

学校读者我要写书评

暂无评论

Leveraging Segment-Anything model for automated zero-shot road width extraction from aerial imagery

Leveraging Segment-Anything model for automated zero-shot ro...

引用

2023 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2023

作者： Xu, Nan Nice, Kerry Seneviratne, Sachith Stevenson, Mark Faculty of Engineering and It Department of Infrastructure Engineering Australia Melbourne School of Design Transport Health and Urban Systems Research Lab Australia The University of Melbourne Optimization and Pattern Recognition Group Faculty of Engineering and It ParkvilleVIC Australia

ISBN: (纸本)9798350382204

Segment-Anything model (SAM) is a foundation segmentation model published in April 2023. Trained on an unprecedented 11 million annotated images, the model can generate segmented masks bearing clear-cut contours by integrating user-provided prompts. It is zero-shot transferable, requiring no task-specific training. Nevertheless, its applicability for geographic vision tasks has not been fully evaluated. There is no automated prompt-feeding method incorporating with SAM that can work efficiently for purposeful batch processing as well. To fill these gaps, we developed a process that can be executed automatically from visual-prompts extraction to road width measurement, utilizing OpenStreetMap (OSM) and SAM. By examining the quality of segmentation in various image contexts, we evaluated the capacity and limitations of SAM working on aerial imagery. Through comparing measured widths to VicRoads records, we validated the specially designed width-measuring algorithm for high precision and accuracy. After this process, prompt-indicated zero-shot approach in solving basic geographic vision tasks is to be shaped synchronously on both theory and application ends. © 2023 IEEE.

关键词： Remote sensing

来源：评论

学校读者我要写书评

暂无评论

GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations

arXiv

引用

arXiv 2023年

作者： Ranasinghe, Nisal Senanayake, Damith Seneviratne, Sachith Premaratne, Malin Halgamuge, Saman AI Optimization and Pattern Recognition Research Group Dept. of Mechanical Eng. University of Melbourne Australia Melbourne School of Design University of Melbourne Australia Department of Electrical and Computer Systems Engineering Monash University Australia

Traditional machine learning is generally treated as a black-box optimization problem and does not typically produce interpretable functions that connect inputs and outputs. However, the ability to discover such interpretable functions is desirable. In this work, we propose GINN-LP, an interpretable neural network to discover the form and coefficients of the underlying equation of a dataset, when the equation is assumed to take the form of a multivariate Laurent Polynomial. This is facilitated by a new type of interpretable neural network block, named the "power-term approximator block", consisting of logarithmic and exponential activation functions. GINN-LP is end-to-end differentiable, making it possible to use backpropagation for training. We propose a neural network growth strategy that will enable finding the suitable number of terms in the Laurent polynomial that represents the data, along with sparsity regularization to promote the discovery of concise equations. To the best of our knowledge, this is the first model that can discover arbitrary multivariate Laurent polynomial terms without any prior information on the order. Our approach is first evaluated on a subset of data used in SRBench, a benchmark for symbolic regression. We first show that GINN-LP outperforms the state-of-the-art symbolic regression methods on datasets generated using 48 real-world equations in the form of multivariate Laurent polynomials. Next, we propose an ensemble method that combines our method with a high-performing symbolic regression method, enabling us to discover non-Laurent polynomial equations. We achieve state-of-the-art results in equation discovery, showing an absolute improvement of 7.1% over the best contender, by applying this ensemble method to 113 datasets within SRBench with known ground-truth equations. © 2023, CC BY-NC-ND.

关键词： Polynomials

来源：评论

学校读者我要写书评

暂无评论

Leveraging Segment-Anything model for automated zero-shot road width extraction from aerial imagery

Leveraging Segment-Anything model for automated zero-shot ro...

引用

Proceedings of the Digital Image Computing: Technqiues and Applications (DICTA)

作者： Nan Xu Kerry Nice Sachith Seneviratne Mark Stevenson Department of Infrastructure Engineering Faculty of Engineering and IT Transport Health and Urban Systems Research Lab Melbourne School of Design Optimization and Pattern Recognition Group Faculty of Engineering and IT The University of Melbourne Parkville Victoria Australia

关键词：

来源：评论

学校读者我要写书评

暂无评论

Manual-Guided Dialogue for Flexible Conversational Agents

arXiv

引用

arXiv 2022年

作者： Takanobu, Ryuichi Zhou, Hao Lin, Yankai Li, Peng Zhou, Jie Huang, Minlie The CoAI Group DCST Institute for Artificial Intelligence State Key Lab of Intelligent Technology and Systems Beijing National Research Center for Information Science and Technology Tsinghua University Beijing100084 China Pattern Recognition Center WeChat AI Tencent Inc. China

How to build and use dialogue data efficiently, and how to deploy models in different domains at scale can be two critical issues in building a task-oriented dialogue system. In this paper, we propose a novel manual-guided dialogue scheme to alleviate these problems, where the agent learns the tasks from both dialogue and manuals. The manual is an unstructured textual document that guides the agent in interacting with users and the database during the conversation. Our proposed scheme reduces the dependence of dialogue models on fine-grained domain ontology, and makes them more flexible to adapt to various domains. We then contribute a fully-annotated multi-domain dataset MagDial to support our scheme. It introduces three dialogue modeling subtasks: instruction matching, argument filling, and response generation. Modeling these subtasks is consistent with the human agent’s behavior patterns. Experiments demonstrate that the manual-guided dialogue scheme improves data efficiency and domain scalability in building dialogue systems. The dataset and benchmark will be publicly available for promoting future research. © 2022, CC BY.

关键词： Speech processing

来源：评论

学校读者我要写书评

暂无评论

Selecting Stickers in Open-Domain Dialogue through Multitask Learning

arXiv

引用

arXiv 2022年

作者： Zhang, Zhexin Zhu, Yeshuang Fei, Zhengcong Zhang, Jinchao Zhou, Jie The CoAI Group DCST China Institute for Artificial Intelligence China State Key Lab of Intelligent Technology and Systems China Beijing National Research Center for Information Science and Technology China Tsinghua University Beijing100084 China Pattern Recognition Center WeChat AI Tencent Inc China

With the increasing popularity of online chatting, stickers are becoming important in our online communication. Selecting appropriate stickers in open-domain dialogue requires a comprehensive understanding of both dialogues and stickers, as well as the relationship between the two types of modalities. To tackle these challenges, we propose a multitask learning method comprised of three auxiliary tasks to enhance the understanding of dialogue history, emotion and semantic meaning of stickers. Extensive experiments conducted on a recent challenging dataset show that our model can better combine the multimodal information and achieve significantly higher accuracy over strong baselines. Ablation study further verifies the effectiveness of each auxiliary task. Our code is available at https://***/nonstopfor/Sticker-Selection. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation

arXiv

引用

arXiv 2022年

作者： Ke, Pei Zhou, Hao Lin, Yankai Li, Peng Zhou, Jie Zhu, Xiaoyan Huang, Minlie The CoAI group DCST Institute for Artificial Intelligence State Key Lab of Intelligent Technology and Systems Beijing National Research Center for Information Science and Technology Tsinghua University Beijing100084 China Pattern Recognition Center WeChat AI Tencent Inc. China Tsinghua University China

Existing reference-free metrics have obvious limitations for evaluating controlled text generation models. Unsupervised metrics can only provide a task-agnostic evaluation result which correlates weakly with human judgments, whereas supervised ones may over-fit task-specific data with poor generalization ability to other datasets. In this paper, we propose an unsupervised reference-free metric called CTRLEval, which evaluates controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks. On top of these tasks, the metric assembles the generation probabilities from a pre-trained language model without any model training. Experimental results show that our metric has higher correlations with human judgments than other baselines, while obtaining better generalization of evaluating generated texts from different models and with different qualities Copyright © 2022, The Authors. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Description-Enhanced Label Embedding Contrastive Learning for Text Classification

arXiv

引用

arXiv 2023年

作者： Zhang, Kun Wu, Le Lv, Guangyi Chen, Enhong Ruan, Shulan Liu, Jing Zhang, Zhiqiang Zhou, Jun Wang, Meng School of Computer and Information Hefei University of Technology Anhui Hefei230029 China AI Lab Lenovo Research Beijing100094 China The School of Computer Science and Technology University of Science and Technology of China Hefei230026 China National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing100190 China Ant Group CO. Ltd Hangzhou310007 China

Text Classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially Pre-trained Language Models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this paper, we employ Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning and extend R2-Net to a novel Description-Enhanced Label Embedding network (DELE) from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on Contrastive Learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that R2-Net can effectively improve classification performance, and DELE can make better

关键词： Classification (of information)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：