检索结果-内蒙古大学图书馆

Correctable Landmark Discovery via large Models for Vision-Language Navigation

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2024年第12期46卷 8534-8548页

作者： Lin, Bingqian Nie, Yunshuang Wei, Ziming Zhu, Yi Xu, Hang Ma, Shikui Liu, Jianzhuang Liang, Xiaodan Sun Yat Sen Univ Shenzhen Campus Shenzhen 518107 Peoples R China Huawei Noahs Ark Lab Shanghai 200120 Peoples R China Dataa Robot Co Beijing 100022 Peoples R China Shenzhen Inst Adv Technol Shenzhen 518055 Peoples R China PengCheng Lab Shenzhen 518066 Peoples R China

Vision-Language Navigation (VLN) requires the agent to follow language instructions to reach a target position. A key factor for successful navigation is to align the landmarks implied in the instruction with diverse visual observations. However, previous VLN agents fail to perform accurate modality alignment especially in unexplored scenes, since they learn from limited navigation data and lack sufficient open-world alignment knowledge. In this work, we propose a new VLN paradigm, called COrrectable LaNdmark DiScOvery via large ModEls (CONSOLE). In CONSOLE, we cast VLN as an open-world sequential landmark discovery problem, by introducing a novel correctable landmark discovery scheme based on two large models ChatGPT and CLIP. Specifically, we use ChatGPT to provide rich open-world landmark cooccurrence commonsense, and conduct CLIP-driven landmark discovery based on these commonsense priors. To mitigate the noise in the priors due to the lack of visual constraints, we introduce a learnable cooccurrence scoring module, which corrects the importance of each cooccurrence according to actual observations for accurate landmark discovery. We further design an observation enhancement strategy for an elegant combination of our framework with different VLN agents, where we utilize the corrected landmark features to obtain enhanced observation features for action decision. Extensive experimental results on multiple popular VLN benchmarks (R2R, REVERIE, R4R, RxR) show the significant superiority of CONSOLE over strong baselines. Especially, our CONSOLE establishes the new state-of-the-art results on R2R and R4R in unseen scenarios.

关键词： Navigation Task analysis visualization Chatbots Artificial intelligence data models Robots Vision-language navigation open-world landmark discovery large language models

来源：评论

学校读者我要写书评

暂无评论

large Language Models-Based Robots for Enterprise Financial Statement Auditing 6

Large Language Models-Based Robots for Enterprise Financial ...

引用

6th International Conference on Machine Learning, Big data and Business Intelligence, MLBDBI 2024

作者： Yang, Yuanqi Zhang, Haichao Shen, Weidong Chen, Haoxuan Cao, Yi Zhao, Liangyu State Grid Jibei Electric Power Company Limited China Beijing Information Science and Technology University China

ISBN: (纸本)9798331541798

This article aims to enhance the digital and intelligent capabilities of enterprise financial statement audits by proposing and constructing optimization strategies and frameworks for audit robots based on large language models. It utilizes large language models to design the audit data robot, audit workpaper robot, and audit analysis robot for human-computer interaction, thereby lowering the technical threshold for auditors and improving their understanding of personalized audit requirements. The audit data robot intelligently extracts data through natural language commands, the audit workpaper robot automates the preparation of workpapers through intent recognition and tool library selection, and the audit analysis robot enhances decision-making levels using visualization and intelligent analysis techniques. The audit robots constructed based on large language models will effectively address existing issues in current audits and improve audit efficiency and intelligence levels. © 2024 ieee.

关键词： Human robot interaction

来源：评论

学校读者我要写书评

暂无评论

Exploring Multidisciplinary data visualization and data analysis with R 2

Exploring Multidisciplinary Data Visualization and Data Anal...

引用

2nd ieee International Conference on Advancements and Key Challenges in Green Energy and Computing, AKGEC 2024

作者： Shahare, Gayatri Manchalwar, Janvi Wankhede, Khushi Meshram, Shreya Kalbande, Kamlesh Wyawahare, Nikhil Nagpur India G H Raisoni College of Engineering Nagpur India

ISBN: (纸本)9798350390179

R is a proprietary programming language that has a large ecosystem of packages for data exploration and visualization, as well as strong community support. This article investigates a wide range of topics, including trend visualizations using the R programming language, sentiment analysis on Twitter, and basic voltage detection and frequency anomalies. It talks about the impact of sentiment analysis on various sensors and presents the PeakValley algorithm for real-time grid event analysis. In order to visualize load profiles and find trends in consumption, Ris employed multiple linear *** cgmquantify package for glucose data in Python and R is evaluated, as is the integration of the FacebookAPI with R for social network *** paper also discusses big data categorization in medicine, R's interface with Hadoop for digital libraries, and Weka, RapidMiner, and R for time series analysis in structural health monitoring. It highlights R's adaptability and recommends that future studies concentrate on improving realtime processing, integrating machine learning, and visualizing techniques. © 2024 ieee.

关键词： data assimilation

来源：评论

学校读者我要写书评

暂无评论

Homeomorphism Prior for False Positive and Negative Problem in Medical Image Dense Contrastive Representation Learning

引用

ieee TRANSACTIONS ON PATTERN analysis AND MACHINE INTELLIGENCE 2025年第5期47卷 4122-4139页

作者： He, Yuting Wang, Boyu Ge, Rongjun Chen, Yang Yang, Guanyu Li, Shuo Southeast Univ Key Lab New Generat Artificial Intelligence Techno Minist Educ Nanjing Peoples R China Case Western Reserve Univ Dept Biomed Engn Cleveland OH 44106 USA Western Univ Dept Comp Sci London ON N6A 3K7 Canada Southeast Univ Sch Instrument Sci & Engn Nanjing 210018 Peoples R China Ctr Rech en Informat Biomedicale Sino Francais CRI Nanjing 210024 Peoples R China Jiangsu Prov Joint Int Res Lab Med Informat Proc Nanjing 210024 Peoples R China Case Western Reserve Univ Dept Biomed Engn Dept Comp & Data Sci Cleveland OH 44106 USA

Dense contrastive representation learning (DCRL) has greatly improved the learning efficiency for image dense prediction tasks, showing its great potential to reduce the large costs of medical image collection and dense annotation. However, the properties of medical images make unreliable correspondence discovery, bringing an open problem of large-scale false positive and negative (FP&N) pairs in DCRL. In this paper, we propose GEoMetric vIsual deNse sImilarity (GEMINI) learning which embeds the homeomorphism prior to DCRL and enables a reliable correspondence discovery for effective dense contrast. We proposes a deformable homeomorphism learning (DHL) which models the homeomorphism of medical images and learns to estimate a deformable mapping to predict the pixels' correspondence under the condition of topological preservation. It effectively reduces the searching space of pairing and drives an implicit and soft learning of negative pairs via gradient. We also proposes a geometric semantic similarity (GSS) which extracts semantic information in features to measure the alignment degree for the correspondence learning. It will promote the learning efficiency and performance of deformation, constructing positive pairs reliably. We implement two practical variants on two typical representation learning tasks in our experiments. Our promising results on seven datasets which outperform the existing methods show our great superiority. We will release our code at a companion website.

关键词： Biomedical imaging Semantics Reliability Topology Feature extraction Representation learning Deformation Computed tomography data mining visualization Medical image analysis dense contrastive representation learning false positive and negative pairs problem homeomorphism prior correspondence problem

来源：评论

学校读者我要写书评

暂无评论

Semantic Understanding of Traffic Scenes with large Vision Language Models 35

Semantic Understanding of Traffic Scenes with Large Vision L...

引用

ieee Intelligent Vehicles symposium (IV)

作者： Jain, Sandesh Thapa, Surendrabikram Chen, Kuan-Ting Abbott, A. Lynn Sarkar, Abhijit Virginia Tech Bradley Dept Elect & Comp Engn Blacksburg VA 24061 USA Virginia Tech Transportat Inst Blacksburg VA 24061 USA

ISBN: (纸本)9798350348811;9798350348828

This paper investigates the integration of large Vision Language Models (LVLMs) with multi-sensor information, including visual and localization data from cameras and LiDAR data to a holistic understanding of traffic videos. Traffic scene understanding is a challenging problem. With complex interaction between the road actors, infrastructure, and traffic rules, it is often difficult to answer questions related to road safety, pedestrian safety, safe maneuvering characteristics, and human factors. Typical processes use a single task-oriented neural network model and combine them through semantic and symbolic reasoning. These processes often suffer from reasoning bias and incompleteness. In recent years, LVLMs have opened new avenues to perceive spatiotemporal information. These models can leverage the large knowledge base from the world and summarize spatiotemporal information effectively. The interactive nature of most of these systems allows humans to directly interact in a visual question-answering mode. In this paper, we have extensively tested the capabilities of such LVLMs to answer key transportation research questions from videos captured through front cameras. We have curated an extensive set of multiple-choice questions to evaluate the performance of these LVLMs. Our results show that LVLMs have abilities to understand various transportation-related aspects to a great extent. Furthermore, we have shown that the addition of supplementary modalities to the VQA settings helps improve the performance of LVLMs. With the addition of 3D trajectories of surrounding objects with the 2D video frames, we observed a significant increase in MCQ performance related to vehicle-to-vehicle interaction tasks. The resources for this paper can be found at https://***/sandeshrjain/lvlm-scene

关键词： large vision language models (LVLM) scene analysis automated perception

来源：评论

学校读者我要写书评

暂无评论

Big data and data visualization Challenges

Big Data and Data Visualization Challenges

引用

2023 ieee International Conference on Big data, Bigdata 2023

作者： Muniswamaiah, Manoj Agerwala, Tilak Tappert, Charles C. Pace University Seidenberg School of CSIS White PlainsNY United States

ISBN: (纸本)9798350324457

Big data has gained attention due to its significant impact on society. With the growth of IoT, data has grown exponentially. Big data has played an essential role in decision-making. After collecting a large volume of data, one of the critical tasks is performing analysis. As a result, there is a rise in the adoption of data visualization, where the raw data is converted into useful information through research and the generation of patterns. The representation of the natural numbers offers an easier way to interpret the collected data, and thus, it becomes easier to comprehend the significance of the data collected. While data visualization is significant, it faces numerous challenges;therefore, this paper aims to provide the different challenges of Big data visualization. © 2023 ieee.

关键词： Big data cloud computing data visualization databases Internet of Things (IoT)

来源：评论

学校读者我要写书评

暂无评论

Visually Grounded Few-Shot Word Learning in Low-Resource Settings

引用

ieee-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2024年 32卷 2544-2554页

作者： Nortje, Leanne Oneata, Dan Kamper, Herman Stellenbosch Univ Dept Elect & Elect Engn ZA-7600 Stellenbosch South Africa Univ Politehn Bucuresti RO-060042 Bucharest Romania

We propose a visually grounded speech model that learns new words and their visual depictions from just a few word-image example pairs. Given a set of test images and a spoken query, we ask the model which image depicts the query word. Previous work has simplified this few-shot learning problem by either using an artificial setting with digit word-image pairs or by using a large number of examples per class. Moreover, all previous studies were performed using English speech-image data. We propose an approach that can work on natural word-image pairs but with less examples, i.e. fewer shots, and then illustrate how this approach can be applied for multimodal few-shot learning in a real low-resource language, Yo'uub'a. Our approach involves using the given word-image example pairs to mine new unsupervised word-image training pairs from large collections of unlabelled speech and images. Additionally, we use a word-to-image attention mechanism to determine word-image similarity. With this new model, we achieve better performance with fewer shots than previous approaches on an existing English benchmark. Many of the model's mistakes are due to confusion between visual concepts co-occurring in similar contexts. The experiments on Yo'uub'a show the benefit of transferring knowledge from a multimodal model trained on a larger set of English speech-image data.

关键词： Task analysis Acoustics visualization Adaptation models Training Speech processing Reviews Few-shot learning multimodal modelling visually grounded speech models word acquisition low-resource language

来源：评论

学校读者我要写书评

暂无评论

A Survivor in the Era of large-Scale Pretraining: An Empirical Study of One-Stage Referring Expression Comprehension

引用

ieee TRANSACTIONS ON MULTIMEDIA 2024年 26卷 3689-3700页

作者： Luo, Gen Zhou, Yiyi Sun, Jiamu Sun, Xiaoshuai Ji, Rongrong Xiamen Univ Key Lab Multimedia Trusted Percept & Efficient Com Minist Educ China Fujian 361005 Peoples R China Peng Cheng Lab Shenzhen 518000 Peoples R China

One-stage Referring Expression Comprehension (REC) is a task that requires accurate alignment between text descriptions and visual content. In recent years, numerous efforts have been devoted to cross-modal learning for REC, while the influence of other factors in this task still lacks a systematic study. To fill this gap, we conduct an empirical study in this article. Concretely, we ablate 42 candidate designs/settings based on a common REC framework, and these candidates cover the entire process of one-stage REC from network design to model training. Afterwards, we conduct over 100 experimental trials on three REC benchmark datasets. The extensive experimental results reveal the key factors that affect REC performance in addition to multi-modal fusion, e.g., multi-scale features and data augmentation. Based on these findings, we further propose a simple yet strong model called SimREC, which achieves new state-of-the-art performance on these benchmarks. In addition to these progresses, we also find that with much less training overhead and parameters, SimREC can achieve better performance than a set of large-scale pre-trained models, e.g., UNITER and VILLA, portraying the special role of REC in existing V&L research.

关键词： Task analysis visualization Training Head Cognition Systematics Sun Computer vision object recognition

来源：评论

学校读者我要写书评

暂无评论

A NOVEL CLOUD-NATIVE AND MULTI-PLATFORM PARALLELIZED SBAS-INSAR ALGORITHM

A NOVEL CLOUD-NATIVE AND MULTI-PLATFORM PARALLELIZED SBAS-IN...

引用

ieee International Geoscience and Remote Sensing symposium (IGARSS)

作者： Yu, Peichen Wang, Chao Tang, Yixian Zhang, Weikang You, Haihang Guan, Shaoyang Zou, Lichuan Zhang, Hong Chinese Acad Sci Aerosp Informat Res Inst Key Lab Digital Earth Sci Beijing 100094 Peoples R China Int Res Ctr Big Data Sustainable Dev Goals Beijing 100094 Peoples R China Chinese Acad Sci Inst Comp Technol Beijing 100190 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China

ISBN: (纸本)9798350360332;9798350360325

Facing massive Synthetic Aperture Radar (SAR) data, the challenge of achieving rapid and efficient data processing has garnered attention. Currently, most time-series InSAR processing solutions are designed to operate on a single computing platform, leading to drawbacks such as inflexible deployment and slow data transmission. Leveraging the open-source Kubernetes platform, we have employed cloud-native technology to deploy a cross-platform, multi-level parallelization algorithm across multiple nodes. The algorithm is designed based on modular concept, and its modules can successfully achieve cross-platform deployment by relying on both the Network File System (NFS) protocol and the Common Internet File System (CFIS) protocol. This overcomes the limitations of the previous data file storage system, which solely depended on the NFS protocol, leading to deployment on a single operating system. Deploying the algorithm on different operating systems, our results show that the speeds of Linux platform's algorithm parallel modules were improved 16% and 30%, respectively. The successful cross-platform operation of the algorithm enables the data processing workflow to assimilate the strengths of different operating platforms, enhancing data visualization capabilities and gaining support from diverse platform resources. This introduces a new approach for large-scale time-series InSAR processing.

关键词： Cloud native SAR SBAS Interferometry Parallelization

来源：评论

学校读者我要写书评

暂无评论

Demo: Cohort visualization and analysis of Patients with Inflammatory Bowel Disease

Demo: Cohort Visualization and Analysis of Patients with Inf...

引用

14th ieee Workshop on Visual Analytics in Healthcare (VAHC)

作者： Sessler, David Ahmad, Salmah Kohlhammer, Jorn Fraunhofer IGD Darmstadt Germany Tech Univ Darmstadt Darmstadt Germany

ISBN: (纸本)9798350330243

This demo paper introduces the final version of a cohort analysis module for the support of treating patients with inflammatory bowel disease (IBD). It is not trivial to correctly diagnose the specific IBD in patients, and wrongly treated patients have to endure the disease effects for a long time, with large costs for the individuals and the healthcare systems. The goal of this work is complementing the examination of individual patients with interactive analyses of cohorts and populations with similar disease patterns to support learning from such similarities for future treatments. We report on additional data and functionality compared to 2021 and discuss an evaluation with eight IBD experts.

关键词： Human-centered computing visualization Visual Analytics Applied computing Life and medical sciences Health care information systems

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：