检索结果-内蒙古大学图书馆

Do LLMs Really Adapt to Domains? An Ontology Learning Perspective 23rd

学校读者我要写书评

暂无评论

Do LLMs Really Adapt to Domains? An Ontology Learning Persp...

23rd International Semantic web Conference, ISWC 2024

作者： Mai, Huu Tan Chu, Cuong Xuan Paulheim, Heiko University of Mannheim Data and Web Science Group Mannheim68131 Germany Bosch Center for AI Renningen71272 Germany

ISBN: (纸本)9783031778438

Large Language Models (LLMs) have demonstrated unprecedented prowess across various natural language processing tasks in various application domains. Recent studies show that LLMs can be leveraged to perform lexical semantic tasks, such as Knowledge Base Completion (KBC) or Ontology Learning (OL). However, it has not effectively been verified whether their success is due to their ability to reason over unstructured or semi-structured data, or their effective learning of linguistic patterns and senses alone. This unresolved question is particularly crucial when dealing with domain-specific data, where the lexical senses and their meaning can completely differ from what a LLM has learned during its training stage. This paper investigates the following question: Do LLMs really adapt to domains and remain consistent in the extraction of structured knowledge, or do they only learn lexical senses instead of reasoning? To answer this question and, we devise a controlled experiment setup that uses WordNet to synthesize parallel corpora, with English and gibberish terms. We examine the differences in the outputs of LLMs for each corpus in two OL tasks: relation extraction and taxonomy discovery. Empirical results show that, while adapting to the gibberish corpora, off-the-shelf LLMs do not consistently reason over semantic relationships between concepts, and instead leverage senses and their frame. However, fine-tuning improves the performance of LLMs on lexical semantic tasks even when the domain-specific terms are arbitrary and unseen during pre-training, hinting at the applicability of pre-trained LLMs for OL. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Domain Knowledge

Evaluating Knowledge Generation and Self-Refinement Strategies for LLM-based Column Type Annotation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Korini, Keti Bizer, Christian Data and Web Science Group University of Mannheim Mannheim Germany

Understanding the semantics of columns in relational tables is an important pre-processing step for indexing data lakes in order to provide rich data search. An approach to establishing such understanding is column type annotation (CTA) where the goal is to annotate table columns with terms from a given vocabulary. This paper experimentally compares different knowledge generation and self-refinement strategies for LLM-based column type annotation. The strategies include using LLMs to generate term definitions, error-based refinement of term definitions, self-correction, and finetuning using examples and term definitions. We evaluate these strategies along two dimensions: effectiveness measured as F1 performance and efficiency measured in terms of token usage and cost. Our experiments show that the best performing strategy depends on the model/dataset combination. We find that using training data to generate label definitions outperforms using the same data as demonstrations for in-context learning for two out of three datasets using OpenAI models. The experiments further show that using the LLMs to refine label definitions brings an average increase of 3.9% F1 in 10 out of 12 setups compared to the performance of the non-refined definitions. Combining fine-tuned models with self-refined term definitions results in the overall highest performance, outperforming zero-shot prompting fine-tuned models by at least 3% in F1 score. The costs analysis shows that while reaching similar F1 score, self-refinement via prompting is more cost efficient for use cases requiring smaller amounts of tables to be annotated while fine-tuning is more efficient for large amounts of tables. Copyright © 2025, The Authors. All rights reserved.

关键词： Semantics

Machine learning-based detection of concept drift in business processes

学校读者我要写书评

暂无评论

Process science 2025年第1期2卷 1-26页

作者： Kraus, Alexander van der Aa, Han Data and Web Science Group University of Mannheim Mannheim Germany Faculty of Computer Science University of Vienna Vienna Austria

Concept drift in process mining occurs when a single event log includes data from multiple versions of a process, making the detection of such drifts essential for ensuring reliable process mining results. Although many techniques have been proposed, they exhibit limitations in accuracy and scope. Specifically, their accuracy diminishes when facing noise, varying drift types, or different levels of change severity. Additionally, these techniques primarily focus on detecting sudden and gradual drifts, overlooking the automated detection of incremental and recurring drifts. To address these limitations, we present CV4CDD-4D, a novel approach for automated concept drift detection that can identify sudden, gradual, incremental, and recurring drifts. Our approach follows an entirely different paradigm. Specifically, it employs a supervised machine learning model fine-tuned on a large collection of event logs with known concept drifts, enabling the model to learn how drifts manifest in event logs. The possibility to train such a model has recently emerged through a tool that generates event logs with known concept drifts. However, applying supervised machine learning remains challenging due to the complexities of encoding. To address this, we propose converting an event log into an image-based representation that captures process evolution over time, enabling the use of a state-of-the-art computer vision model to detect drifts. Our experiments show that our approach, compared to existing solutions, improves the accuracy and robustness to noise of drift detection while extending coverage to a broader range of drift types, highlighting the potential of this new paradigm.

关键词：

CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks 24

学校读者我要写书评

暂无评论

CosPGD: an efficient white-box adversarial attack for pixel-...

Proceedings of the 41st International Conference on Machine Learning

作者： Shashank Agnihotri Steffen Jung Margret Keuper Data and Web Science Group University of Mannheim Germany Max-Planck-Institute for Informatics Saarland Informatics Campus Germany and Data and Web Science Group University of Mannheim Germany Data and Web Science Group University of Mannheim Germany and Max-Planck-Institute for Informatics Saarland Informatics Campus Germany

While neural networks allow highly accurate predictions in many tasks, their lack of robustness towards even slight input perturbations often hampers their deployment. Adversarial attacks such as the seminal projected gradient descent (PGD) offer an effective means to evaluate a model's robustness and dedicated solutions have been proposed for attacks on semantic segmentation or optical flow estimation. While they attempt to increase the attack's efficiency, a further objective is to balance its effect, so that it acts on the entire image domain instead of isolated point-wise predictions. This often comes at the cost of optimization stability and thus efficiency. Here, we propose CosPGD, an attack that encourages more balanced errors over the entire image domain while increasing the attack's overall efficiency. To this end, CosPGD leverages a simple alignment score computed from any pixel-wise prediction and its target to scale the loss in a smooth and fully differentiable way. It leads to efficient evaluations of a model's robustness for semantic segmentation as well as regression models (such as optical flow, disparity estimation, or image restoration), and it allows it to outperform the previous SotA attack on semantic segmentation. We provide code for the CosPGD algorithm and example usage at https://***/shashankskagnihotri/cospgd.

关键词：

Self-Refinement Strategies for LLM-based Product Attribute Value Extraction

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Brinkmann, Alexander Bizer, Christian University of Mannheim Data & Web Science Group Schloss Mannheim68161 Germany

Structured product data, in the form of attribute-value pairs, is essential for e-commerce platforms to support features such as faceted product search and attribute-based product comparison. However, vendors often provide unstructured product descriptions, making attribute value extraction necessary to ensure data consistency and usability. Large language models (LLMs) have demonstrated their potential for product attribute value extraction in few-shot scenarios. Recent research has shown that self-refinement techniques can improve the performance of LLMs on tasks such as code generation and text-to-SQL translation. For other tasks, the application of these techniques has resulted in increased costs due to processing additional tokens, without achieving any improvement in performance. This paper investigates applying two self-refinement techniques — error-based prompt rewriting and self-correction — to the product attribute value extraction task. The self-refinement techniques are evaluated across zero-shot, few-shot in-context learning, and fine-tuning scenarios using GPT-4o. The experiments show that both self-refinement techniques fail to significantly improve the extraction performance while substantially increasing processing costs. For scenarios with development data, fine-tuning yields the highest performance, while the ramp-up costs of fine-tuning are balanced out as the amount of product descriptions increases. Copyright © 2025, The Authors. All rights reserved.

关键词： Costs

Discovering multi-agent systems for resource-centric business process simulation

学校读者我要写书评

暂无评论

Process science 2025年第1期2卷 1-26页

作者： Kirchdorfer, Lukas Blümel, Robert Kampik, Timotheus van der Aa, Han Stuckenschmidt, Heiner SAP Signavio Walldorf Germany Data and Web Science Group University of Mannheim Mannheim Germany Faculty of Computer Science University of Vienna Vienna Austria Department of Computing Science Umeå University Umeå Sweden

Business process simulation (BPS) is a powerful tool for estimating process performance across different scenarios, offering critical support for organizational process redesign and optimization. Traditional BPS approaches predominantly rely on a control-flow-first perspective by enriching a process model with simulation parameters. While these approaches seem suitable for capturing centrally orchestrated processes, such as those managed by workflow systems, they fall short of accurately reflecting real-world processes characterized by decentralized decision-making and distinct resource behaviors. To overcome this limitation, we propose AgentSimulator, a resource-first BPS approach that discovers a multi-agent system from an event log. By modeling the distinct behaviors and interaction patterns of individual resources, AgentSimulator effectively simulates the underlying process. Our approach automatically identifies whether resource behavior is rather orchestrated or autonomous, adapting to the specific decision-making structure of the process. Experimental results reveal that AgentSimulator achieves state-of-the-art simulation accuracy while ensuring high adaptability to various process types.

关键词：

GUIDE: LLM-Driven GUI Generation Decomposition for Automated Prototyping

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Kolthoff, Kristian Kretzer, Felix Bartelt, Christian Maedche, Alexander Ponzetto, Simone Paolo Institute for Software and Systems Engineering Clausthal University of Technology Clausthal Germany Human-Centered Systems Lab Karlsruhe Institute of Technology Karlsruhe Germany Data and Web Science Group University of Mannheim Mannheim Germany

Graphical user interface (GUI) prototyping serves as one of the most valuable techniques for enhancing the elicitation of requirements, facilitating the visualization and refinement of customer needs and closely integrating the customer into the development activities. While GUI prototyping has a positive impact on the software development process, it simultaneously demands significant effort and resources. The emergence of Large Language Models (LLMs) with their impressive code generation capabilities offers a promising approach for automating GUI prototyping. Despite their potential, there is a gap between current LLM-based prototyping solutions and traditional user-based GUI prototyping approaches which provide visual representations of the GUI prototypes and direct editing functionality. In contrast, LLMs and related generative approaches merely produce text sequences or non-editable image output, which lacks both mentioned aspects and therefore impede supporting GUI prototyping. Moreover, minor changes requested by the user typically lead to an inefficient regeneration of the entire GUI prototype when using LLMs directly. In this work, we propose GUIDE, a novel LLM-driven GUI generation decomposition approach seamlessly integrated into the popular prototyping framework Figma. Our approach initially decomposes high-level GUI descriptions into fine-granular GUI requirements, which are subsequently translated into Material Design GUI prototypes, enabling higher controllability and more efficient adaption of changes. To efficiently conduct prompting-based generation of Material Design GUI prototypes, we propose a retrieval-augmented generation (RAG) approach to integrate the component library. Our preliminary evaluation demonstrates the effectiveness of GUIDE in bridging the gap between LLM generation capabilities and traditional GUI prototyping workflows, offering a more effective and controlled user-based approach to LLM-driven GUI prototyping. Video presentation

关键词： Graphical user interfaces

Multi-dataset and Transfer Learning Using Gene Expression Knowledge Graphs

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sousa, Rita T. Paulheim, Heiko Data and web Science Group University of Mannheim Germany

Gene expression datasets offer insights into gene regulation mechanisms, biochemical pathways, and cellular functions. Additionally, comparing gene expression profiles between disease and control patients can deepen the understanding of disease pathology. Therefore, machine learning has been used to process gene expression data, with patient diagnosis emerging as one of the most popular applications. Although gene expression data can provide valuable insights, challenges arise because the number of patients in expression datasets is usually limited, and the data from different datasets with different gene expressions cannot be easily combined. This work proposes a novel methodology to address these challenges by integrating multiple gene expression datasets and domain-specific knowledge using knowledge graphs, a unique tool for biomedical data integration. Then, vector representations are produced using knowledge graph embedding techniques, which are used as inputs for a graph neural network and a multi-layer perceptron. We evaluate the efficacy of our methodology in three settings: single-dataset learning, multi-dataset learning, and transfer learning. The experimental results show that combining gene expression datasets and domain-specific knowledge improves patient diagnosis in all three settings. © 2025, CC BY.

关键词： Knowledge graph

Deep Learning for Climate Action: Computer Vision Analysis of Visual Narratives on X

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Prasse, Katharina Kleinmann, Marcel Adam, Inken Beckersjürgen, Kerstin Edte, Andreas Frroku, Jona Gumpp, Timotheus Jung, Steffen Bravo, Isaac Walter, Stefanie Keuper, Margret Data & Web Science Group University of Mannheim Mannheim Germany School of Governance Technical University of Munich Munich Germany University of Mannheim Max Planck Institute for Informatics Saarland Informatics Campus Saarbrücken Mannheim Germany

Climate change is one of the most pressing challenges of the 21st century, sparking widespread discourse across social media platforms. Activists, policymakers, and researchers seek to understand public sentiment and narratives while access to social media data has become increasingly restricted in the post-API era. In this study, we analyze a dataset of climate change-related tweets from X (formerly Twitter) shared in 2019, containing 730k tweets along with the shared images. Our approach integrates statistical analysis, image classification, object detection, and sentiment analysis to explore visual narratives in climate discourse. Additionally, we introduce a graphical user interface (GUI) to facilitate interactive data exploration. Our findings reveal key themes in climate communication, highlight sentiment divergence between images and text, and underscore the strengths and limitations of foundation models in analyzing social media imagery. By releasing our code and tools, we aim to support future research on the intersection of climate change, social media, and computer vision. © 2025, CC BY-SA.

关键词： Tweets