检索结果-内蒙古大学图书馆

LMEye: An Interactive Perception Network for large Language Models

ieee TRANSACTIONS ON MULTIMEDIA 2024年 26卷 10952-10964页

作者： Li, Yunxin Hu, Baotian Chen, Xinyu Ma, Lin Xu, Yong Zhang, Min Harbin Inst Technol Dept Comp Sci & Technol Shenzhen 518000 Peoples R China Meituan Beijing 100102 Peoples R China

Current efficient approaches to building Multimodal large Language Models (MLLMs) mainly incorporate visual information into LLMs with a simple visual mapping network such as a linear projection layer, a multilayer perceptron (MLP), or Q-former from BLIP-2. Such networks project the image feature once and do not consider the interaction between the image and the human inputs. Hence, the obtained visual information without being connected to human intention may be inadequate for LLMs to generate intention-following responses, which we refer to as static visual information. To alleviate this issue, our paper introduces LMEye, a human-like eye with a play-and-plug interactive perception network, designed to enable dynamic interaction between LLMs and external visual information. It can allow the LLM to request the desired visual information aligned with various human instructions, which we term dynamic visual information acquisition. Specifically, LMEye consists of a simple visual mapping network to provide the basic perception of an image for LLMs. It also contains additional modules responsible for acquiring requests from LLMs, performing request-based visual information seeking, and transmitting the resulting interacted visual information to LLMs, respectively. In this way, LLMs act to understand the human query, deliver the corresponding request to the request-based visual information interaction module, and generate the response based on the interleaved multimodal information. We evaluate LMEye through extensive experiments on multimodal benchmarks, demonstrating that it significantly improves zero-shot performances on various multimodal tasks compared to previous methods, with fewer parameters. Moreover, we also verify its effectiveness and scalability on various language models and video understanding, respectively.

关键词： visualization Task analysis data models Tuning large language models Training Cognition Multimodal large language models (MLLMs) visual-language learning interactive perception network

来源：评论

学校读者我要写书评

暂无评论

A visualization METHOD FOR GPR data INTERPRETATION AND TARGET analysis

A VISUALIZATION METHOD FOR GPR DATA INTERPRETATION AND TARGE...

引用

ieee International Geoscience and Remote Sensing symposium (IGARSS)

作者： Jin, Kaisheng Gao, Yesheng Liu, Xingzhao Shanghai Jiao Tong Univ Dept Elect Engn Shanghai Peoples R China

ISBN: (数字)9781665427920

ISBN: (纸本)9781665427920

In this paper, a novel visualization method for GPR data interpretation and target analysis is proposed for underground target detection and recognition. After preprocessing, the original B-scan image has been transformed into two binary images containing potential targets. Then in the target extraction part, a novel row connection clustering algorithm is applied to separate all possible hyperbolic regions. Finally, a neural network is used to analyze the extracted slices and to estimate the parameters including the material and size of the targets. To illustrate the performance of the proposed method, this paper uses synthetic data generated from gprMax, which demonstrates the exciting accuracy of target detection and recognition.

关键词： Row connection clustering algorithm ground penetrating radar (GPR) neural network signal processing method visualization

来源：评论

学校读者我要写书评

暂无评论

High Performance visualization for Astrophysics and Cosmology 33

High Performance Visualization for Astrophysics and Cosmolog...

引用

33rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2025

作者： Tuccari, Nicola Sciacca, Eva Vitello, Fabio Colonnelli, Iacopo Becerra, Yolanda Cintero, Enric Sosa Marin, Guillermo Jaros, Milan Riha, Lubomir Strakos, Petr Trujillo-Gomez, Sebastian Tramontana, Emiliano Wissing, Robert Istituto Nazionale di Astrofisica Catania Italy Università di Torino Torino Italy Barcelona Supercomputing Center Barcelona Spain IT4Innovations National Supercomputing Center Ostrava Czech Republic Heidelberg Institute for Theoretical Studies Heidelberg Germany Università degli Studi di Catania Italy University of Oslo Institute of Theoretical Astrophysics Oslo Norway

ISBN: (纸本)9798331524937

Modern Astrophysics and Cosmology (A&C) projects produce immense data volumes, necessitating advanced software tools for data access, storage, and analysis. visualization Interface for the Virtual Observatory (VisIVO) is one such tool enabling multi-dimensional data analysis and knowledge discovery across complex astrophysical datasets. Leveraging containerization and virtualization, VisIVO has been deployed on various distributed computing platforms. Additionally, Blender, an open-source 3D suite, provides robust tools for rendering and processing volumetric data, making it suitable for visualizing complex datasets. At the SPACE Center of Excellence these tools are being adapted for high-performance visualization of cosmological simulations performed with GADGET and ChaNGa on pre-exascale systems. However, implementing high-performance visualization on diverse HPC platforms presents several challenges, including hardware and software compatibility, data management, scalability, performance portability, and efficient resource allocation. This paper outlines strategies to integrate VisIVO with workflow frameworks and streaming platforms to address these challenges. Workflow frameworks enhance portability, scheduling, and reproducibility of visualization workflows on pre-exascale systems used in A&C simulations. We also discuss the use of streaming platforms to enable concurrent (i.e. in-situ) analysis and visualization of simulations, reducing the need to store full simulation data by leveraging distributed databases that stream the output data in real time. Lastly, we present an adaptation of Blender to handle large-scale particle-based astrophysical data, offering high-quality visualization with interactive exploration capabilities. © 2025 ieee.

关键词： Dark Matter

来源：评论

学校读者我要写书评

暂无评论

Learning Depth Representation From RGB-D Videos by Time-Aware Contrastive Pre-Training

引用

ieee TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2024年第6期34卷 4143-4158页

作者： He, Zongtao Wang, Liuyi Dang, Ronghao Li, Shu Yan, Qingqing Liu, Chengju Chen, Qijun Tongji Univ Robot & Artificial Intelligence Lab RAIL Shanghai 201804 Peoples R China

Existing end-to-end depth representation in embodied AI is often task-specific and lacks the benefits of emerging pre-training paradigm due to limited datasets and training techniques for RGB-D videos. To address the challenge of obtaining robust and generalized depth representation for embodied AI, we introduce a unified RGB-D video dataset (UniRGBD) and a novel time-aware contrastive (TAC) pre-training approach. UniRGBD addresses the scarcity of large-scale depth pre-training datasets by providing a comprehensive collection of data from diverse sources in a unified format, enabling convenient data loading and accommodating various data domains. We also design an RGB-Depth alignment evaluation procedure and introduce a novel Near-K accuracy metric to assess the scene understanding capability of the depth encoder. Then, the TAC pre-training approach fills the gap in depth pre-training methods suitable for RGB-D videos by leveraging the intrinsic similarity between temporally proximate frames. TAC incorporates a soft label design that acts as valid label noise, enhancing the depth semantic extraction and promoting diverse and generalized knowledge acquisition. Furthermore, the adjustments in perspective between temporally proximate frames facilitate the extraction of invariant and comprehensive features, enhancing the robustness of the learned depth representation. Additionally, the inclusion of temporal information stabilizes training gradients and enables spatio-temporal depth perception. Comprehensive evaluation of RGB-Depth alignment demonstrates the superiority of our approach over state-of-the-art methods. We also conduct uncertainty analysis and a novel zero-shot experiment to validate the robustness and generalization of the TAC approach. Moreover, our TAC pre-training demonstrates significant performance improvements in various embodied AI tasks, providing compelling evidence of its efficacy across diverse domains.

关键词： Task analysis Artificial intelligence Videos Training databases visualization Feature extraction Depth representation pre-training methods contrastive learning embodied AI

来源：评论

学校读者我要写书评

暂无评论

Recursive Least-Squares Estimator-Aided Online Learning for Visual Tracking

引用

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2024年第3期46卷 1881-1897页

作者： Gao, Jin Lu, Yan Qi, Xiaojuan Kou, Yutong Li, Bing Li, Liang Yu, Shan Hu, Weiming Chinese Acad Sci Inst Automat Natl Lab Pattern Recognit Beijing 100190 Peoples R China Microsoft Res Asia Beijing 100080 Peoples R China Univ Hong Kong Dept Elect & Elect Engn Hong Kong Peoples R China Chinese Acad Sci Inst Automat Beijing 100190 Peoples R China

Tracking visual objects from a single initial exemplar in the testing phase has been broadly cast as a one-/few-shot problem, i.e., one-shot learning for initial adaptation and few-shot learning for online adaptation. The recent few-shot online adaptation methods incorporate the prior knowledge from large amounts of annotated training data via complex meta-learning optimization in the offline phase. This helps the online deep trackers to achieve fast adaptation and reduce overfitting risk in tracking. In this paper, we propose a simple yet effective recursive least-squares estimator-aided online learning approach for few-shot online adaptation without requiring offline training. It allows an in-built memory retention mechanism for the model to remember the knowledge about the object seen before, and thus the seen data can be safely removed from training. This also bears certain similarities to the emerging continual learning field in preventing catastrophic forgetting. This mechanism enables us to unveil the power of modern online deep trackers without incurring too much extra computational cost. We evaluate our approach based on two networks in the online learning families for tracking, i.e., multi-layer perceptrons in RT-MDNet and convolutional neural networks in DiMP. The consistent improvements on several challenging tracking benchmarks demonstrate its effectiveness and efficiency.

关键词： visualization Training Adaptation models data models Optimization Task analysis Robustness Online learning few-shot online adaptation visual tracking continual learning recursive least-squares estimation

来源：评论

学校读者我要写书评

暂无评论

LOOPS: LOcally Optimized Polygon Simplification

引用

COMPUTER GRAPHICS FORUM 2022年第3期41卷 355-365页

作者： Amiraghdam, Alireza Diehl, Alexandra Pajarola, Renato Univ Zurich Dept Informat Zurich Switzerland

Displaying polygonal vector data is essential in various application scenarios such as geometry visualization, vector graphics rendering, CAD drawing and in particular geographic, or cartographic visualization. Dealing with static polygonal datasets that has a large scale and are highly detailed poses several challenges to the efficient and adaptive display of polygons in interactive geographic visualization applications. For linear vector data, only recently a GPU-based level-of-detail (LOD) polyline simplification and rendering approach has been presented which can perform locally-adaptive LOD visualization of large-scale line datasets interactively. However, locally optimized LOD simplification and interactive display of large-scale polygon data, consisting of filled vector line loops, remains still a challenge, specifically in 3D geographic visualizations where varying LOD over a scene is necessary. Our solution to this challenge is a novel technique for locally-optimized simplification and visualization of 2D polygons over a 3D terrain which features a parallelized point-inside-polygon testing mechanism. Our approach is capable of employing any simplification algorithm that sequentially removes vertices such as Douglas-Peucker and Wang-Muller. Moreover, we generalized our technique to also visualizing polylines in order to have a unified method for displaying both data types. The results and performance analysis show that our new algorithm can handle large datasets containing polygons composed of millions of segments in real time, and has a lower memory demand and higher performance in comparison to prior methods of line simplification and visualization.

关键词： CCS Concepts • Human-centered computing → Geographic visualization visualization techniques • Theory of computation → Computational geometry • Computing methodologies → Rendering Rasterization

来源：评论

学校读者我要写书评

暂无评论

visualization of Hidden Patterns in WhatsApp Chat using Sentimental analysis 3

Visualization of Hidden Patterns in WhatsApp Chat using Sent...

引用

3rd International Conference on Intelligent data Communication Technologies and Internet of Things, IDCIoT 2025

作者： Vinora, A. Suryakumar, L.S.S. Harivishnu, T.S. Jaspin, K. Selvaraj, Arikumar K. Nancy Deborah, M. Velammal College of Engineering and Technology Madurai India St. Joseph's Institute of Technology Chennai India SRM Institute of Science and Technology Chennai India

ISBN: (纸本)9798331527549

Challenges include handling unstructured chat data, ensuring privacy and ethical standards, managing large data volumes, accurately interpreting sentiment despite context, sarcasm, and irony, and creating effective visualizations. The proposed paper aims to develop a robust chat analyzer for WhatsApp conversations, leveraging Python libraries like pandas, matplotlib, seaborn, and natural language processing techniques. By filtering out irrelevant messages and group notifications, the tool focuses on meaningful interactions and employs sentiment analysis to provide insights into user dynamics. This comprehensive analyzer offers visualization and statistical analysis of chat data through a user-friendly application deployed on the Heroku web platform. The project improves comprehension of WhatsApp messages by integrating machine learning and NLP, allowing users to extract important data and handle unwanted content effectively. The suggested procedure involves utilizing a machine learning algorithm that combines VADER sentiment analysis for accurate emotion detection with keyword frequency analysis for trend identification. The new system shows a 15% boost in accuracy and a 20% increase in processing efficiency for sentiment analysis and user activity detection when compared to current models. © 2025 ieee.

关键词： data visualization matplotlib Natural language processing (NLP) Python libraries (pandas seaborn) Sentiment analysis User behavior insights VADER WhatsApp chat analyzer

来源：评论

学校读者我要写书评

暂无评论

Self-Supervised Tracking via Target-Aware data Synthesis

引用

ieee TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024年第7期35卷 9186-9197页

作者： Li, Xin Pei, Wenjie Wang, Yaowei He, Zhenyu Lu, Huchuan Yang, Ming-Hsuan Peng Cheng Lab Shenzhen 518055 Peoples R China Harbin Inst Technol Sch Comp Sci & Technol Shenzhen 518055 Peoples R China Dalian Univ Technol Sch Informat & Commun Engn Dalian 116024 Peoples R China Univ Calif Merced Dept Elect Engn & Comp Sci Merced CA 95344 USA

While deep-learning-based tracking methods have achieved substantial progress, they entail large-scale and high-quality annotated data for sufficient training. To eliminate expensive and exhaustive annotation, we study self-supervised (SS) learning for visual tracking. In this work, we develop the crop-transform-paste operation, which is able to synthesize sufficient training data by simulating various appearance variations during tracking, including appearance variations of objects and background interference. Since the target state is known in all synthesized data, existing deep trackers can be trained in routine ways using the synthesized data without human annotation. The proposed target-aware data-synthesis method adapts existing tracking approaches within a SS learning framework without algorithmic changes. Thus, the proposed SS learning mechanism can be seamlessly integrated into existing tracking frameworks to perform training. Extensive experiments show that our method: 1) achieves favorable performance against supervised (Su) learning schemes under the cases with limited annotations;2) helps deal with various tracking challenges such as object deformation, occlusion (OCC), or background clutter (BC) due to its manipulability;3) performs favorably against the state-of-the-art unsupervised tracking methods;and 4) boosts the performance of various state-of-the-art Su learning frameworks, including SiamRPN++, DiMP, and TransT.

关键词： Target tracking Training visualization Annotations Training data Learning systems Task analysis Crop-transform-paste self-supervised (SS) learning visual tracking

来源：评论

学校读者我要写书评

暂无评论

Lossy Compression for visualization of Atmospheric data 11

Lossy Compression for Visualization of Atmospheric Data

引用

ieee 11th symposium on large data analysis and visualization (LDAV)

作者： Semeraro, Dave Orf, Leigh Texas Adv Comp Ctr Austin TX 78758 USA Univ Wisconsin Madison WI USA

ISBN: (纸本)9781665432832

Lossy compression is a data compression technique that sacrifices precision for the sake of higher compression rates. While loss of precision is unacceptable when storing simulation data for check pointing, it has little discernable impact on visualization. Saving simulation output for later examination is still a prevalent workflow. Domain scientists often return to data from older runs to examine the data in a new context. Storage of visualization data at full precision is not necessary for this purpose. The use of lossy compression can therefore relieve the pressure on HPC storage equipment or be used to store data at higher temporal resolution than without compression. In this poster we show how lossy compression was used to store visualization data for the analysis of a supercell thunderstorm. The visual results will be shown as well as details of how the compression was used in the workflow.

关键词： data Compression Atmospheric science

来源：评论

学校读者我要写书评

暂无评论

A Survey of visualization and analysis in High-Resolution Connectomics

引用

COMPUTER GRAPHICS FORUM 2022年第3期41卷 573-607页

作者： Beyer, Johanna Troidl, Jakob Boorboor, Saeed Hadwiger, Markus Kaufman, Arie Pfister, Hanspeter Harvard Univ Visual Comp Grp Cambridge MA 02138 USA SUNY Stony Brook Dept Comp Sci Stony Brook NY 11794 USA King Abdullah Univ Sci & Technol Visual Comp Ctr Thuwal Saudi Arabia

The field of connectomics aims to reconstruct the wiring diagram of Neurons and synapses to enable new insights into the workings of the brain. Reconstructing and analyzing the Neuronal connectivity, however, relies on many individual steps, starting from high-resolution data acquisition to automated segmentation, proofreading, interactive data exploration, and circuit analysis. All of these steps have to handle large and complex datasets and rely on or benefit from integrated visualization methods. In this state-of-the-art report, we describe visualization methods that can be applied throughout the connectomics pipeline, from data acquisition to circuit analysis. We first define the different steps of the pipeline and focus on how visualization is currently integrated into these steps. We also survey open science initiatives in connectomics, including usable open-source tools and publicly available datasets. Finally, we discuss open challenges and possible future directions of this exciting research field.

关键词： CCS Concepts • Human-centered computing → visualization systems and tools Scientific visualization • Applied computing → Life and medical sciences Systems biology

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：