检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Paul, Sayan Roychoudhury, Ruddra Dev Bhowmick, Brojeshwar Visual Computing and Embodied AI Lab TCS Research India

visual odometry (VO) is essential for enabling accurate pointgoal navigation of embodied agents in indoor environments where GPS and compass sensors are unreliable and inaccurate. However, traditional VO methods face challenges in wide-baseline scenarios, where fast robot motions and low frames per second (FPS) during inference hinder their performance, leading to drift and catastrophic failures in point-goal navigation. Recent deep-learned VO methods show robust performance but suffer from sample inefficiency during training;hence, they require huge datasets and compute resources. So, we propose a robust and sampleefficient VO pipeline based on motion priors available while an agent is navigating an environment. It consists of a training-free action-prior based geometric VO module that estimates a coarse relative pose which is further consumed as a motion prior by a deep-learned VO model, which finally produces a fine relative pose to be used by the navigation policy. This strategy helps our pipeline achieve up to 2x sample efficiency during training and demonstrates superior accuracy and robustness in point-goal navigation tasks compared to state-of-the-art VO method(s). Realistic indoor environments of the Gibson dataset is used in the AIHabitat simulator to evaluate the proposed approach using navigation metrics (like success/SPL) and pose metrics (like RPE/ATE). We hope this method further opens a direction of work where motion priors from various sources can be utilized to improve VO estimates and achieve better results in embodied navigation tasks. © 2024, CC BY-NC-ND.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Task Planning for Object Rearrangement in Multi-room Environments

arXiv

引用

arXiv 2024年

作者： Mirakhor, Karan Ghosh, Sourav Das, Dipanjan Bhowmick, Brojeshwar Visual Computing and Embodied Intelligence Lab TCS Research Kolkata India

Object rearrangement in a multi-room setup should produce a reasonable plan that reduces the agent's overall travel and the number of steps. Recent state-of-the-art methods fail to produce such plans because they rely on explicit exploration for discovering unseen objects due to partial observability and a heuristic planner to sequence the actions for rearrangement. This paper proposes a novel hierarchical task planner to efficiently plan a sequence of actions to discover unseen objects and rearrange misplaced objects within an untidy house to achieve a desired tidy state. The proposed method introduces several novel techniques, including (i) a method for discovering unseen objects using commonsense knowledge from large language models, (ii) a collision resolution and buffer prediction method based on Cross-Entropy Method to handle blocked goal and swap cases, (iii) a directed spatial graphbased state space for scalability, and (iv) deep reinforcement learning (RL) for producing an efficient planner. The planner interleaves the discovery of unseen objects and rearrangement to minimize the number of steps taken and overall traversal of the agent. The paper also presents new metrics and a benchmark dataset called MoPOR to evaluate the effectiveness of the rearrangement planning in a multi-room setting. The experimental results demonstrate that the proposed method effectively addresses the multi-room rearrangement problem. © 2024, CC BY.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Image Retrieval with Hard Negative Centers and Quadruplet Center Loss in Deep Hashing

Image Retrieval with Hard Negative Centers and Quadruplet Ce...

引用

Future Machine Learning and Data Science (FMLDS), IEEE International Conference on

作者： Nikolaos Konstantinou Theodoros Semertzidis Anastasios Dimou Petros Daras The Visual Computing Lab Centre for Research and Technology Hellas Thessaloniki Greece

ISBN: (数字)9798350391213

ISBN: (纸本)9798350391220

In this work, we enhance image retrieval performance by building upon the Central Similarity Quantization method in deep learning, which uses 'hash centers'—points in a K-dimensional Hamming space fulfilling certain distance criteria. Our approach is inspired by the concept of selecting hard negatives in deep learning, aiming to boost model performance. We introduce an alternative strategy for quadruplet loss training that employs corresponding positive/negative hash centers instead of positive/negative samples. For each sample, we compute the Hamming distances between its hash code and all hash centers and select the two closest centers as negative centers. The positive center is its random assigned center and is used to guide the sample towards convergence. We then define a quadruplet loss function that encourages the learned hash codes to be both discriminative and compact, minimizing the distances between the sample and its positive center while maximizing the distances between the sample and the negative centers. Our method is evaluated using mAP and Recall@K metrics on the Hotels50k, Stanford Dogs, and CUB200 datasets, demonstrating promising results in image retrieval tasks by effectively utilizing hard negative centers and quadruplet center loss.

关键词： Measurement Deep learning Training Learning systems Codes Quantization (signal) Scalability Image retrieval Dogs Hamming distances

来源：评论

学校读者我要写书评

暂无评论

Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Models 38

Lumen: Unleashing Versatile Vision-Centric Capabilities of L...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Jiao, Yang Chen, Shaoxiang Jie, Zequn Chen, Jingjing Ma, Lin Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Meituan China

Large Multimodal Model (LMM) is a hot research topic in the computer vision area and has also demonstrated remarkable potential across multiple disciplinary fields. A recent trend is to further extend and enhance the perception capabilities of LMMs. The current methods follow the paradigm of adapting the visual task outputs to language-oriented formats. This adaptation leads to the convenient development of such LMMs with minimal modifications, however, it overlooks the inductive biases within diverse visual tasks and hinders the learning of perception capabilities. To address this issue, we propose a novel LMM architecture named Lumen, which decouples the learning of perception capabilities into task-agnostic and task-specific stages. Firstly, Lumen promotes fine-grained vision-language concept alignment, which is the fundamental capability for various visual tasks. Thus the output of the task-agnostic stage is a shared representation for all vision-centric tasks we address in this paper. Afterward, the task-specific decoding is carried out by flexibly routing the shared representation to lightweight task decoders with negligible training efforts. Comprehensive experimental results on a series of vision-centric and VQA benchmarks indicate that our Lumen model not only achieves or surpasses the performance of existing LMM-based approaches in a range of vision-centric tasks while maintaining general visual understanding and instruction following capabilities. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

DeepStack: Deeply Stacking visual Tokens is Surprisingly Simple and Effective for LMMs 38

DeepStack: Deeply Stacking Visual Tokens is Surprisingly Sim...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Meng, Lingchen Yang, Jianwei Tian, Rui Dai, Xiyang Wu, Zuxuan Gao, Jianfeng Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Microsoft Corporation United States

Most large multimodal models (LMMs) are implemented by feeding visual tokens as a sequence into the first layer of a large language model (LLM). The resulting architecture is simple but significantly increases computation and memory costs, as it has to handle a large number of additional tokens in its input layer. This paper presents a new architecture DeepStack for LMMs. Considering N layers in the language and vision transformer of LMMs, we stack the visual tokens into N groups and feed each group to its aligned transformer layer from bottom to top, as illustrated in Fig. 1. Surprisingly, this simple method greatly enhances the power of LMMs to model interactions among visual tokens across layers but with minimal additional cost. We apply DeepStack to both language and vision transformer in LMMs, and validate the effectiveness of DeepStack LMMs with extensive empirical results. Using the same context length, our DeepStack 7B and 13B parameters surpass their counterparts by 2.7 and 2.9 on average across 9 benchmarks, respectively. Using only one-fifth of the context length, DeepStack rivals closely to the counterparts that use the full context length. These gains are particularly pronounced on high-resolution tasks, e.g., 4.2, 11.0, and 4.0 improvements on TextVQA, DocVQA, and InfoVQA compared to LLaVA-1.5-7B, respectively. We further apply DeepStack to vision transformer layers, which brings us a similar amount of improvements, 3.8 on average compared with LLaVA-1.5-7B. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

The Digital Ocean lab (DOL): Centralising Ocean Research in the Baltic Sea

The Digital Ocean Lab (DOL): Centralising Ocean Research in ...

引用

OCEANS 2024 - Halifax, OCEANS 2024

作者： Umlauft, Alexandru Schmidt, Felix Lünsdorf, Gerrit Globisch, Torben Daske, Nele Suhrbier, Patrick Bauer, Kristine Krohmann, Sascha Von Lukas, Uwe Freiherr Maritime Graphics University of Rostock Rostock Germany Visual Computing Fraunhofer Igd Rostock Germany Maritime Graphics Fraunhofer Igd Rostock Germany Digital Ocean Lab Fraunhofer Igd Rostock Germany

ISBN: (纸本)9798331540081

Concentrating several underwater research applications into one research station directly on the sea can prove to be a cost-saving method and foster more research on maritime domains. We aim to develop a maritime research test bed in the Baltic Sea and create digital twins of the environment. These digital twins can greatly help to better understand the oceans and react to the challenges of climate change. Here we describe our technical setup and experience with maintaining this maritime research field ('DOL') and the issues we encountered. © 2024 IEEE.

关键词： digital twin ocean observation sensors underwater

来源：评论

学校读者我要写书评

暂无评论

OmniTokenizer: A Joint Image-Video Tokenizer for visual Generation 38

OmniTokenizer: A Joint Image-Video Tokenizer for Visual Gene...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Wang, Junke Jiang, Yi Yuan, Zehuan Peng, Binyue Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Bytedance Inc. China

Tokenizer, serving as a translator to map the intricate visual data into a compact latent space, lies at the core of visual generative models. Based on the finding that existing tokenizers are tailored to image or video inputs, this paper presents OmniTokenizer, a transformer-based tokenizer for joint image and video tokenization. OmniTokenizer is designed with a spatial-temporal decoupled architecture, which integrates window and causal attention for spatial and temporal modeling. To exploit the complementary nature of image and video data, we further propose a progressive training strategy, where OmniTokenizer is first trained on image data on a fixed resolution to develop the spatial encoding capacity and then jointly trained on image and video data on multiple resolutions to learn the temporal dynamics. OmniTokenizer, for the first time, handles both image and video inputs within a unified framework and proves the possibility of realizing their synergy. Extensive experiments demonstrate that OmniTokenizer achieves state-of-the-art (SOTA) reconstruction performance on various image and video datasets, e.g., 1.11 reconstruction FID on ImageNet and 42 reconstruction FVD on UCF-101, beating the previous SOTA methods by 13% and 26%, respectively. Additionally, we also show that when integrated with OmniTokenizer, both language model-based approaches and diffusion models can realize advanced visual synthesis performance, underscoring the superiority and versatility of our method. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Active Learning Inspired ControlNet Guidance for Augmenting Semantic Segmentation Datasets

arXiv

引用

arXiv 2025年

作者： Kniesel, Hannah Hermosilla, Pedro Ropinski, Timo Visual Computing Group Ulm University Germany Computer Vision Lab TU Vienna Austria

Recent advances in conditional image generation from diffusion models have shown great potential in achieving impressive image quality while preserving the constraints introduced by the user. In particular, ControlNet enables precise alignment between ground truth segmentation masks and the generated image content, allowing the enhancement of training datasets in segmentation tasks. This raises a key question: Can ControlNet additionally be guided to generate the most informative synthetic samples for a specific task? Inspired by active learning, where the most informative real-world samples are selected based on sample difficulty or model uncertainty, we propose the first approach to integrate active learning-based selection metrics into the backward diffusion process for sample generation. Specifically, we explore uncertainty, query by committee, and expected model change, which are commonly used in active learning, and demonstrate their application for guiding the sample generation process through gradient approximation. Our method is training-free, modifying only the backward diffusion process, allowing it to be used on any pretrained ControlNet. Using this process, we show that segmentation models trained with guided synthetic data outperform those trained on non-guided synthetic data. Our work underscores the need for advanced control mechanisms for diffusion-based models, which are not only aligned with image content but additionally downstream task performance, highlighting the true potential of synthetic data generation. Copyright © 2025, The Authors. All rights reserved.

关键词： Active learning

来源：评论

学校读者我要写书评

暂无评论

Retrieval Augmented Recipe Generation

Retrieval Augmented Recipe Generation

引用

2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025

作者： Liu, Guoshan Yin, Hailong Zhu, Bin Chen, Jingjing Ngo, Chong-Wah Jiang, Yu-Gang School of Computer Science Fudan University Shanghai Key Lab of Intelligent Information Processing China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Singapore Management University Singapore

ISBN: (纸本)9798331510831

The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method - first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs), which have achieved notable success across a variety of vision and language tasks, shed light on generating both ingredients and instructions directly from images. Nevertheless, LMMs still face the common issue of hallu-cinations during recipe generation, leading to suboptimal performance. To tackle this issue, we propose a retrieval augmented large multimodal model for recipe generation. We first introduce Stochastic Diversified Retrieval Augmentation (SDRA) to retrieve recipes semantically related to the image from an existing datastore as a supplement, integrating them into the prompt to add diverse and rich context to the input image. Additionally, Self-Consistency Ensemble Voting mechanism is proposed to determine the most confident prediction recipes as the final output. It calculates the consistency among generated recipe candidates, which use different retrieval recipes as context for generation. Extensive experiments validate the effectiveness of our proposed method, which demonstrates state-of-the-art (SOTA) performance in recipe generation on the Recipe1M dataset. © 2025 IEEE.

关键词： Food ingredients

来源：评论

学校读者我要写书评

暂无评论

Electroencephalogram (EEG) Brain Signals to Detect Alcoholism Based on Deep Learning

引用

Computers, Materials & Continua 2021年第6期67卷 3329-3348页

作者： Emad-ul-Haq Qazi Muhammad Hussain Hatim A.AboAlsamh Visual Computing Lab Department of Computer ScienceCollege of Computer and Information SciencesKing Saud UniversityRiyadh11543Saudi Arabia

The detection of alcoholism is of great importance due to its effects on individuals and *** alcoholism detection system(AADS)based on electroencephalogram(EEG)signals is effective,but the design of a robust AADS is a challenging ***’current designs are based on conventional,hand-engineered methods and restricted *** by the excellent deep learning(DL)success in many recognition tasks,we implement an AAD system based on EEG signals using DL.A DL model requires huge number of learnable parameters and also needs a large dataset of EEG signals for training which is not easy to obtain for the AAD *** order to solve this problem,we propose a multi-channel Pyramidal neural convolutional(MP-CNN)network that requires a less number of learnable *** the deep CNN model,we build an AAD system to detect from EEG signal segments whether the subject is alcoholic or *** validate the robustness and effectiveness of proposed AADS using KDD,a benchmark dataset for alcoholism detection *** order to find the brain region that contributes significant role in AAD,we investigated the effects of selected 19 EEG channels(SC-19),those from the whole brain(ALL-61),and 05 brain regions,i.e.,TEMP,OCCIP,CENT,FRONT,and *** results show that SC-19 contributes significant role in AAD with the accuracy of 100%.The comparison reveals that the state-of-the-art systems are outperformed by the *** proposed AADS will be useful in medical diagnosis research and health care systems.

关键词： Electroencephalogram convolutional neural network deep learning alcoholism

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：