检索结果-内蒙古大学图书馆

Aligning, Autoencoding and Prompting large Language Models for Novel Disease Reporting

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2025年第5期47卷 3332-3343页

作者： Liu, Fenglin Wu, Xian Huang, Jinfa Yang, Bang Branson, Kim Schwab, Patrick Clifton, Lei Zhang, Ping Luo, Jiebo Zheng, Yefeng Clifton, David A. Univ Oxford Dept Engn Sci Oxford OX3 7DQ England Jarvis Res Ctr Tencent You Tu Lab Shenzhen 518000 Peoples R China Univ Rochester Dept Comp Sci Rochester NY 14627 USA Peking Univ Sch Elect & Comp Engn Shenzhen 518055 Peoples R China Peng Cheng Lab Shenzhen 518052 Peoples R China Glaxo SmithKline London SG1 2NY England Univ Oxford Nuffield Dept Primary Care Hlth Sci Appl Digital Hlth ADH Oxford OX1 2JD England Ohio State Univ Dept Comp Sci & Engn Dept Biomed Informat Columbus OH 43210 USA Westlake Univ Med Artificial Intelligence Lab Hangzhou 310024 Peoples R China Oxford Suzhou Ctr Adv Res Suzhou 215123 Peoples R China

Given radiology images, automatic radiology report generation aims to produce informative text that reports diseases. It can benefit current clinical practice in diagnostic radiology. Existing methods typically rely on large-scale medical datasets annotated by clinicians to train desirable models. However, for novel diseases, sufficient training data are typically not available. We propose a prompt-based deep learning framework, i.e., PromptLLM, to align, autoencode, and prompt the (large) language model to generate reports for novel diseases accurately and efficiently. Our method includes three major steps: 1) aligning visual images and textual reports to learn general knowledge across modalities from diseases where labeled data are sufficient, 2) autoencoding the LLM using unlabeled data of novel diseases to learn the specific knowledge and writing styles of the novel disease, and 3) prompting the LLM with learned knowledge and writing styles to report the novel diseases contained in the radiology images. Through the above three steps, with limited labels on novel diseases, we show that PromptLLM can rapidly learn the corresponding knowledge for accurate novel disease reporting. The experiments on COVID-19 and diverse thorax diseases show that our approach, utilizing 1% of the training data, achieves desirable performance compared to previous methods. It shows that our approach allows us to relax the reliance on labeled data that is common to existing methods. It could have a real-world impact on data analysis during the early stages of novel diseases.

关键词： Diseases Radiology Autoencoders visualization Thorax Dictionaries Decoding Deep learning COVID-19 Training data large language models novel disease diagnosis medical report generation few-shot learning

来源：评论

学校读者我要写书评

暂无评论

RelationLMM: large Multimodal Model as Open and Versatile Visual Relationship Generalist

引用

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2025年第5期47卷 3515-3529页

作者： Xie, Chi Liang, Shuang Li, Jie Zhang, Zhao Zhu, Feng Zhao, Rui Wei, Yichen Tongji Univ Sch Software Engn Shanghai 200070 Peoples R China Sensetime Res Hong Kong Peoples R China Shukun Technol A-11010 Vienna Austria

Visual relationships are crucial for visual perception and reasoning, and cover tasks like Scene Graph Generation, Human-Object Interaction, and object affordance. Despite significant efforts, this field still suffers from the following limitations: specialists for a specific task without considering similar ones, strict and complex task formulations with limited flexibility, and underexploited reasoning with language and knowledge. To solve these limitations, we seek to build a new framework, one model for all tasks, over large Multimodal Models (LMMs). LMMs offer the potential of unifying tasks, flexible forms, and reasoning with language. However, they fail to handle visual relationship tasks well. We find the obstacles include the conflicts between different tasks and insufficient instance-level information. We solve these problems by reforming the data for LMMs, rather than architectures, considering their strong language-in language-out capability. We propose to disassemble tasks into simple and common sub-tasks, verbally estimate instance confidence, and augment instance diversity, all without additional modules. These strategies help us build a visual relationship generalist, RelationLMM, with a simple architecture. Exhaustive experiments demonstrate RelationLMM is strong, generalizable and flexible to different tasks, with one model and one suite of weight.

关键词： visualization Affordances Object detection Detectors Computational modeling Cognition Computer architecture Benchmark testing Training Pipelines Computer vision scene understanding visual relationship detection large multi-modal model

来源：评论

学校读者我要写书评

暂无评论

Visualizing Program Behavior: A Study of Enhanced Program Diagrams Using LLM 54

Visualizing Program Behavior: A Study of Enhanced Program Di...

引用

54th ieee Frontiers in Education Conference, FIE 2024

作者： Li, Ying Yang, Runze Gui, ShiJie Shi, Peng Huang, Xuefei Yang, Da Zhang, Xiaozhou Gai, Yiming School of Computer Science and Engineering Beihang University Beijing China International Innovation Institute Beihang University Key Laboratory of Data Science and Intelligent Computing Hangzhou China

ISBN: (纸本)9798350351507

This paper aims to address the difficulties faced by novice programmers in grasping code structure and execution flow, improving programming thinking, and pinpointing code errors with accuracy. It proposes providing students with program behavior diagrams based on large language models (LLMs) and visualization techniques to achieve personalized guidance. Specifically, these program behavior diagrams include programming thinking visualization diagrams and code vulnerability visualization diagrams. A programming thinking visualization diagram employs static code analysis to gather code structure information, combined with the structured chain-of-thought method to collectively optimize the LLM. This enables the LLM to explain each interpretable part of the code from top to bottom, detailing the programming concepts, and displaying them on a modularized code structure diagram. The code vulnerability visualization diagram primarily utilizes the fine-tuned LLM, optimizing it based on program analysis and clustering analysis methods to accurately identify vulnerabilities in student code and display them on a code flow diagram. Its feature is to visually display to students the error location, error information, and the impact of errors on program flow, rather than providing the programming answers. Lastly, through experiments and statistical analysis of actual teaching data, this paper serves a demonstration that the enhanced models used in the visualization diagram generation process have a noticeable effect on mainstream LLMs, and that visualization diagrams hold significant value for students at different stages of learning. © 2024 ieee.

关键词： Students

来源：评论

学校读者我要写书评

暂无评论

Gander: The Preliminary Design and Evaluation of an AR+Tablet System for Geospatial analysis

Gander: The Preliminary Design and Evaluation of an AR+Table...

引用

2024 ieee International symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2024

作者： Hu, Sathaporn Raza, Muhammad Reilly, Derek Dalhousie University Canada Algoma University Canada

ISBN: (纸本)9798331506919

We explore the combination of headworn augmented reality (AR) displays and handheld tablet devices to support geospatial analysis. In this paper, we present the design of an AR+tablet prototype named Gander. Gander supports the selection of attributes when building a predictive model for some geospatial phenomenon, such as water pollution is correlated to the proximity of large cities, and the comparison of these models. We conducted a walkthrough evaluation with five experts in geospatial analysis and GIS tools. The experts identified strengths and weaknesses in Gander's design. We propose new design changes such as the pancake plot, rank-based visualization, and reducing steps on the tablet interface. © 2024 ieee.

关键词： River pollution

来源：评论

学校读者我要写书评

暂无评论

Dual-View data Hallucination With Semantic Relation Guidance for Few-Shot Image Recognition

引用

ieee TRANSACTIONS ON MULTIMEDIA 2024年 26卷 11302-11315页

作者： Wu, Hefeng Ye, Guangzhi Zhou, Ziyang Tian, Ling Wang, Qing Lin, Liang Sun Yat Sen Univ Sch Comp Sci & Engn Guangzhou 510275 Peoples R China Univ Elect Sci & Technol China Chengdu 610054 Peoples R China

Learning to recognize novel concepts from just a few image samples is very challenging as the learned model is easily overfitted on the few data and results in poor generalizability. One promising but underexplored solution is to compensate for the novel classes by generating plausible samples. However, most existing works of this line exploit visual information only, rendering the generated data easy to be distracted by some challenging factors contained in the few available samples. Being aware of the semantic information in the textual modality that reflects human concepts, this work proposes a novel framework that exploits semantic relations to guide dual-view data hallucination for few-shot image recognition. The proposed framework enables generating more diverse and reasonable data samples for novel classes through effective information transfer from base classes. Specifically, an instance-view data hallucination module hallucinates each sample of a novel class to generate new data by employing local semantic correlated attention and global semantic feature fusion derived from base classes. Meanwhile, a prototype-view data hallucination module exploits semantic-aware measure to estimate the prototype of a novel class and the associated distribution from the few samples, which thereby harvests the prototype as a more stable sample and enables resampling a large number of samples. We conduct extensive experiments and comparisons with state-of-the-art methods on several popular few-shot benchmarks to verify the effectiveness of the proposed framework.

关键词： Semantics visualization Prototypes Task analysis data models Image recognition Few shot learning Few-shot learning data hallucination semantic relation guidance image recognition

来源：评论

学校读者我要写书评

暂无评论

AppMAIS Simple data visualization App

AppMAIS Simple Data Visualization App

引用

ieee SoutheastCon Conference

作者： Richardson, Logan Campell, Christopher Tashakkori, Rahman O'Brien, Will Davis, Sarah Elizabeth Appalachian State Univ Boone NC 28608 USA

ISBN: (纸本)9781665476119

Due to the fact that a large number of honey bee hives have been lost in recent years, there have been some efforts to collect various data to assist with monitoring the hives and determine their health status. The Appalachian Multi-purpose Apiary Informatics Systems (AppMAIS) was funded by the North Carolina General Assembly in 2020 to establish a collaborative research project to monitor the health of honey bee hives. The AppMAIS Simple data visualization App greatly increases our ability to analyze data, detect patterns, and train our ML swarm-like detection models. The app utilizes tooltips, hyperlinking, and highlight functionalities to significantly improve the efficiency of data analysis.

关键词： Precision apiculture data visualization Swarm detection Robbery Dash

来源：评论

学校读者我要写书评

暂无评论

CGVIZ: A CESIUM-BASED visualization SYSTEM FOR MULTI-SOURCE GEOHAZARDS data

CGVIZ: A CESIUM-BASED VISUALIZATION SYSTEM FOR MULTI-SOURCE ...

引用

ieee International Geoscience and Remote Sensing symposium (IGARSS)

作者： Li, Jiabao Huang, Xiaohui Yan, Jining Dong, Yusen Zhang, Junqiang Wang, Lizhe China Univ Geosci Sch Comp Sci 68 JinCheng St Wuhan 430078 Hubei Peoples R China

ISBN: (数字)9781665427920

ISBN: (纸本)9781665427920

Visualizing the associated data of urban geological disasters can better describe the urban spatial information and provide technical support for analyzing geological disasters and upper-level decision-making. Aiming at the multi-source data with various types and complex structures in the urban space, how to integrate, organize, and visualize them is a critical technical problem for the urban geological disaster big data system. This paper deeply researched and discussed the visualization technology and implementation methods of multiple data types related to urban geological disasters. At the same time, we developed a visualization analysis system for urban geological disasters, which realized the integrated visualization of multi-source data in urban space and simulation visualization of geological disasters process. In addition, it can also provide users with related analysis functions of urban geological disasters based on the visualization system.

关键词： Multi-source data Urban geological disasters Disaster process simulation visualization

来源：评论

学校读者我要写书评

暂无评论

A Machine-Learning-Aided Visual analysis Workflow for Investigating Air Pollution data 15

A Machine-Learning-Aided Visual Analysis Workflow for Invest...

引用

ieee 15th Pacific visualization symposium (PacificVis)

作者： Kuo, Yun-Hsin Fujiwara, Takanori Chou, Charles C-K Chen, Chun-houh Ma, Kwan-Liu Univ Calif Davis Davis CA 95616 USA Acad Sinica Taipei Taiwan

ISBN: (纸本)9781665423359

Analyzing air pollution data is challenging as there are various analysis focuses from different aspects: feature (what), space (where), and time (when). As in most geospatial analysis problems, besides high-dimensional features, the temporal and spatial dependencies of air pollution induce the complexity of performing analysis. Machine learning methods, such as dimensionality reduction, can extract and summarize important information of the data to lift the burden of understanding such a complicated environment. In this paper, we present a methodology that utilizes multiple machine learning methods to uniformly explore these aspects. With this methodology, we develop a visual analytic system that supports a flexible analysis workflow, allowing domain experts to freely explore different aspects based on their analysis needs. We demonstrate the capability of our system and analysis workflow supporting a variety of analysis tasks with multiple use cases.

关键词： Visual analytics machine learning analysis workflow dimensionality reduction matrix factorization air pollution

来源：评论

学校读者我要写书评

暂无评论

Gender in 30 Years of ieee visualization

引用

ieee TRANSACTIONS ON visualization AND COMPUTER GRAPHICS 2022年第1期28卷 497-507页

作者： Tovanich, Natkamon Dragicevic, Pierre Isenberg, Petra Paris Saclay IRT SystemX F-91120 Palaiseau France Univ Paris Saclay LISN INRIA CNRS F-91405 Orsay France

We present an exploratory analysis of gender representation among the authors, committee members, and award winners at the ieee visualization (VIS) conference over the last 30 years. Our goal is to provide descriptive data on which diversity discussions and efforts in the community can build. We look in particular at the gender of VIS authors as a proxy for the community at large. We consider measures of overall gender representation among authors, differences in careers, positions in author lists, and collaborations. We found that the proportion of female authors has increased from 9% in the first five years to 22% in the last five years of the conference. Over the years, we found the same representation of women in program committees and slightly more women in organizing committees. Women are less likely to appear in the last author position, but more in the middle positions. In terms of collaboration patterns, female authors tend to collaborate more than expected with other women in the community. All non-gender related data is available on https://***/ydfj4/ and the gender-author matching can be accessed through https://***/volume/1301.

关键词： Engineering profession data visualization Manuals Conferences Collaboration Cleaning Gender issues visualization gender diversity publication scientometry collaboration

来源：评论

学校读者我要写书评

暂无评论

Hades: A Context-Aware Active Storage Framework for Accelerating large-Scale data analysis 24

Hades: A Context-Aware Active Storage Framework for Accelera...

引用

24th ieee/ACM International symposium on Cluster, Cloud, and Internet Computing (CCGrid)

作者： Cernuda, Jaime Logan, Luke Gainaru, Ana Klasky, Scott Lofstead, Jay Kougkas, Anthony Sun, Xian-He IIT Chicago IL 60616 USA Oak Ridge Natl Lab Oak Ridge TN USA Sandia Natl Labs Albuquerque NM USA

ISBN: (纸本)9798350395679;9798350395662

Modern simulation workflows generate and analyze massive amounts of data using I/O libraries like Adios2 and NetCDF. Although extensive work has optimized the I/O processes during the simulation phase, executing analytical queries-which often require iterative traversals of large files for insights-is cumbersome and usually constrained by low I/O performance. Instead of waiting for the analysis phase to process queries, quantities can be derived asynchronously during data production and cached, speeding up future queries. In this work, we introduce a context-aware I/O layer named 'Hades.' It is designed to efficiently derive insights from selected quantities without compromising overall workflow performance. Hades actively and asynchronously computes and stores these quantities while the data is in transit. Hades leverages a hierarchical buffering system with data access-aware prefetching to ensure quick and timely access to relevant data. It offers a flexible query interface empowering users to easily define derived quantities and provide control over data placement decisions. Hades is implemented using an Adios2 plugin engine and the Hermes buffering platform, enabling transparent use by any Adios-powered application or workflow. Experimental results demonstrate performance improvements by up to 3-4x for tested real-world scientific producer-consumer workflows.

关键词： Active Storage Hierarchical Storage Context Awareness Metadata Management data Operator In-transit Computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：