检索结果-内蒙古大学图书馆

A User-Friendly Framework for Generating Model-Preferred Prompts in Text-to-Image Synthesis

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hei, Nailei Guo, Qianyu Wang, Zihao Wang, Yan Wang, Haofen Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Tongji University China College of Design and Innovation Tongji University China

Well-designed prompts have demonstrated the potential to guide text-to-image models in generating amazing images. Although existing prompt engineering methods can provide high-level guidance, it is challenging for novice users to achieve the desired results by manually entering prompts due to a discrepancy between novice-user-input prompts and the model-preferred prompts. To bridge the distribution gap between user input behavior and model training datasets, we first construct a novel Coarse-Fine Granularity Prompts dataset (CFP) and propose a novel User-Friendly Fine-Grained Text Generation framework (UF-FGTG) for automated prompt optimization. For CFP, we construct a novel dataset for text-to-image tasks that combines coarse and fine-grained prompts to facilitate the development of automated prompt generation methods. For UF-FGTG, we propose a novel framework that automatically translates user-input prompts into model-preferred prompts. Specifically, we propose a prompt refiner that continually rewrites prompts to empower users to select results that align with their unique needs. Meanwhile, we integrate image-related loss functions from the text-to-image model into the training process of text generation to generate model-preferred prompts. Additionally, we propose an adaptive feature extraction module to ensure diversity in the generated results. Experiments demonstrate that our approach is capable of generating more visually appealing and diverse images than previous state-of-the-art methods, achieving an average improvement of 5% across six quality and aesthetic metrics. Data and code are available at https://***/Naylenv/UF-FGTG. Copyright © 2024, The Authors. All rights reserved.

关键词： Image enhancement

Time Series-based Active Labeling Framework for Curating a Multispectral Sentinel 2 Imagery Dataset for Crop Type Mapping

学校读者我要写书评

暂无评论

Time Series-based Active Labeling Framework for Curating a M...

IEEE International Symposium on Geoscience and Remote Sensing (IGARSS)

作者： V. Mehmood R. Murtaza Z. Zafar M. Shahzad K. Berns M. M. Fraz School of Electrical Engineering and Computer Science (SEECS) National University of Sciences and Technology (NUST) Islamabad Pakistan Data Science in Earth Observations Technical University of Munich Germany Robotics Research Lab Technical University of Kaiserslautern Germany

Acquiring ground-truth data for crop mapping is a challenging task in developing countries. Limited resources, inconsistent agricultural practices across the country, and inadequate infrastructure at the administrative level pose a significant challenge in collecting dataset information. This study proposes an active labelling framework for automated ground truth data generation in areas with limited or no ground truth information available. Sentinel 2 images and vegetative indices are visually interpreted for the identification of Wheat and Rice fields. This hand-labelled data is incorporated into the framework as training data to generate crop maps at 10m resolution using light weight, ConvLSTM model. Testing is performed for two spatially distinct districts with different field sizes and crop distribution in Pakistan. The results are consistent with an accuracy of 77.8%, F1 score of 87.3% and IOU of 70.7% for Gujranwala and 76%, 82% and 70% for Sargodha, promising the applicability of the proposed approach to generate large-scale crop labels, thereby enhancing the efficiency of crop mapping efforts in developing countries

关键词：

Segregator: Global Point Cloud Registration with Semantic and Geometric Cues

学校读者我要写书评

暂无评论

Segregator: Global Point Cloud Registration with Semantic an...

IEEE International Conference on robotics and Automation (ICRA)

作者： Pengyu Yin Shenghai Yuan Haozhi Cao Xingyu Ji Shuyang Zhang Lihua Xie Centre for Advanced Robotics Technology Innovation (CARTIN) School of Electrical and Electronic Engineering Nanyang Technological University Singapore Department of Electronic and Computer Engineering Hong Kong University of Science and Technology Kowloon Hong Kong SAR China

This paper presents Segregator, a global point cloud registration framework that exploits both semantic information and geometric distribution to efficiently build up outlier-robust correspondences and search for inliers. Current state-of-the-art algorithms rely on point features to set up putative correspondences and refine them by employing pair-wise distance consistency checks. However, such a scheme suffers from degenerate cases, where the descriptive capability of local point features downgrades, and unconstrained cases, where length-preserving (1-TRIMs)-based checks cannot sufficiently constrain whether the current observation is consistent with others, resulting in a complexified NP-complete problem to solve. To tackle these problems, on the one hand, we propose a novel degeneracy-robust and efficient corresponding procedure consisting of both instance-level semantic clusters and geometric-level point features. On the other hand, Gaussian distribution-based translation and rotation invariant measurements (G-TRIMs) are proposed to conduct the consistency check and further constrain the problem size. We validated our proposed algorithm on extensive real-world data-based experiments. The code is available: https://***/Pamphlett/Segregator.

关键词：

A Soft Micro-Robotic Catheter for Aneurysm Treatment: A Novel Design and Enhanced Euler-Bernoulli Model with Cross-Section Optimization

学校读者我要写书评

暂无评论

A Soft Micro-Robotic Catheter for Aneurysm Treatment: A Nove...

IEEE International Conference on robotics and Automation (ICRA)

作者： Emanuele Nicotra Chi Cong Nguyen James Davies Phuoc Thien Phan Trung Thien Hoang Bibhu Sharma Adrienne Ji Kefan Zhu Trung Dung Ngo Van Anh Ho Hung Manh La Nigel H. Lovell Thanh Nho Do Graduate School of Biomedical Engineering Faculty of Engineering UNSW Sydney Kensington Campus Australia More-Than-One Robotics Laboratory Faculty of Sustainable Design Engineering University of Prince Edward Island Charlottetown Canada Soft Haptics Laboratory School of Materials Science Japan Advanced Institute of Science and Technology Kawaguchi Saitama Japan Department of Computer Science and Engineering Advanced Robotics and Automation (ARA) Lab University of Nevada Reno NV USA

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

Aneurysms, balloon-like bulges in blood vessels, present a significant health risk due to their potential to rupture, leading to life-threatening internal bleeding. Current treatments often involve delivering embolic materials or metal coils to fill these bulges, occluding them from the pressure of blood flow. However, clinical micro-catheters that deploy embolic materials used today face limitations, primarily their rigidity and the lack of active control over the bending tip of the catheter. This paper introduces a new soft micro-robotics catheter, with diameter of only 0.8 mm, equipped with a hollow channel. With this new design, the new device can induce bending motions at its tip for active steerability to reach desired aneurysm targets and then perform the delivery of embolic materials and tools. To enhance the control and precise navigation during procedures, a robust mathematical model and image processing techniques are also introduced and validated. Experiments are also performed to characterise and validate the model’s accuracy and the steerability and navigation capabilities of the new micro-catheter.

关键词： Performance evaluation Navigation Metals Aneurysm Bending Mathematical models Rigidity

InstaGraM: Instance-level Graph Modeling for Vectorized HD Map Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Shin, Juyeb Jeong, Hyeonjun Rameau, Francois Kum, Dongsuk Robotics Program Korea Advanced Institute of Science and Technology Daejeon34051 Korea Republic of Graduate School of Mobility Korea Advanced Institute of Science and Technology Daejeon34051 Korea Republic of Computer Science department State University of New York - Korea Songdo21985 Korea Republic of

For scalable autonomous driving, a robust map-based localization system, independent of GPS, is fundamental. To achieve such map-based localization, online high-definition (HD) map construction plays a significant role in accurate estimation of the pose. Although recent advancements in online HD map construction have predominantly investigated on vectorized representation due to its effectiveness, they suffer from computational cost and fixed parametric model, which limit scalability. To alleviate these limitations, we propose a novel HD map learning framework that leverages graph modeling. This framework is designed to learn the construction of diverse geometric shapes, thereby enhancing the scalability of HD map construction. Our approach involves representing the map elements as an instance-level graph by decomposing them into vertices and edges to facilitate accurate and efficient end-to-end vectorized HD map learning. Furthermore, we introduce an association strategy using a Graph Neural Network to efficiently handle the complex geometry of various map elements, while maintaining scalability. Comprehensive experiments on public open dataset show that our proposed network outperforms state-of-the-art model by 1.6 mAP. We further showcase the superior scalability of our approach compared to state-of-the-art methods, achieving a 4.8 mAP improvement in long range configuration. Our code is available at https://***/juyebshin/InstaGraM. Copyright © 2023, The Authors. All rights reserved.

关键词： Graph neural networks

VidEgoThink: ASSESSING EGOCENTRIC VIDEO UNDERSTANDING CAPABILITIES FOR EMBODIED AI

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cheng, Sijie Fang, Kechen Yu, YangYang Zhou, Sicheng Li, Bohao Tian, Ye Li, Tingguang Han, Lei Liu, Yang Department of Computer Science and Technology Tsinghua University China Tsinghua University China Department of Mechanical and Industrial Engineering University of Toronto Canada School of Data Science The Chinese University of HongKong Hong Kong Zhili College Tsinghua University China Tencent Robotics X China

Recent advancements in Multi-modal Large Language Models (MLLMs) have opened new avenues for applications in Embodied AI. Building on previous work, EgoThink, we introduce VidEgoThink, a comprehensive benchmark for evaluating egocentric video understanding capabilities. To bridge the gap between MLLMs and low-level control in Embodied AI, we design four key interrelated tasks: video question-answering, hierarchy planning, visual grounding and reward modeling. To minimize manual annotation costs, we develop an automatic data generation pipeline based on the Ego4D dataset, leveraging the prior knowledge and multimodal capabilities of GPT-4o. Three human annotators then filter the generated data to ensure diversity and quality, resulting in the VidEgoThink benchmark. We conduct extensive experiments with three types of models: API-based MLLMs, open-source image-based MLLMs, and open-source video-based MLLMs. Experimental results indicate that all MLLMs, including GPT-4o, perform poorly across all tasks related to egocentric video understanding. These findings suggest that foundation models still require significant advancements to be effectively applied to first-person scenarios in Embodied AI. In conclusion, VidEgoThink reflects a research trend towards employing MLLMs for egocentric vision, akin to human capabilities, enabling active observation and interaction in the complex real-world environments. Copyright © 2024, The Authors. All rights reserved.

关键词： Video analysis

Event-aided Semantic Scene Completion

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Guo, Shangwei Shi, Hao Wang, Song Yin, Xiaoting Yang, Kailun Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics the National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China College of Computer Science and Technology Zhejiang University Hangzhou310027 China

Autonomous driving systems rely on robust 3D scene understanding. Recent advances in Semantic Scene Completion (SSC) for autonomous driving underscore the limitations of RGB-based approaches, which struggle under motion blur, poor lighting, and adverse weather. Event cameras, offering high dynamic range and low latency, address these challenges by providing asynchronous data that complements RGB inputs. We present DSEC-SSC, the first real-world benchmark specifically designed for event-aided SSC, which includes a novel 4D labeling pipeline for generating dense, visibility-aware labels that adapt dynamically to object motion. Our proposed RGB-Event fusion framework, EvSSC, introduces an Event-aided Lifting Module (ELM) that effectively bridges 2D RGB-Event features to 3D space, enhancing view transformation and the robustness of 3D volume construction across SSC models. Extensive experiments on DSEC-SSC and simulated SemanticKITTI-E demonstrate that EvSSC is adaptable to both transformer-based and LSS-based SSC architectures. Notably, evaluations on SemanticKITTI-C demonstrate that EvSSC achieves consistently improved prediction accuracy across five degradation modes and both In-domain and Out-of-domain settings, achieving up to a 52.5% relative improvement in mIoU when the image sensor partially fails. Additionally, we quantitatively and qualitatively validate the superiority of EvSSC under motion blur and extreme weather conditions, where autonomous driving is challenged. The established datasets and our codebase will be made publicly at https://***/Pandapan01/EvSSC. Copyright © 2025, The Authors. All rights reserved.

关键词： Image enhancement

DOZE: A Dataset for Open-Vocabulary Zero-Shot Object Navigation in Dynamic Environments

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ma, Ji Dai, Hongming Mu, Yao Wu, Pengying Wang, Hao Chi, Xiaowei Fei, Yang Zhang, Shanghang Liu, Chang The Department of Advanced Manufacturing and Robotics College of Engineering Peking University Beijing100871 China The State Key Laboratory of Multimedia Information Processing School of Computer Science Peking University Beijing100871 China The Department of Computer Science The University of Hong Kong Pokfulam Hong Kong The Hong Kong University of Science and Technology Clear Water Bay Kowloon Hong Kong

Zero-Shot Object Navigation (ZSON) requires agents to autonomously locate and approach unseen objects in unfamiliar environments and has emerged as a particularly challenging task within the domain of Embodied AI. Existing datasets for developing ZSON algorithms lack consideration of dynamic obstacles, object attribute diversity, and scene texts, thus exhibiting noticeable discrepancies from real-world situations. To address these issues, we propose a Dataset for Open-Vocabulary Zero-Shot Object Navigation in Dynamic Environments (DOZE) that comprises ten high-fidelity 3D scenes with over 18k tasks, aiming to mimic complex, dynamic real-world scenarios. Specifically, DOZE scenes feature multiple moving humanoid obstacles, a wide array of open-vocabulary objects, diverse distinct-attribute objects, and valuable textual hints. Besides, different from existing datasets that only provide collision checking between the agent and static obstacles, we enhance DOZE by integrating capabilities for detecting collisions between the agent and moving obstacles. This novel functionality enables the evaluation of the agents' collision avoidance abilities in dynamic environments. We test four representative ZSON methods on DOZE, revealing substantial room for improvement in existing approaches concerning navigation efficiency, safety, and object recognition accuracy. Our dataset can be found at https://***/. Copyright © 2024, The Authors. All rights reserved.

关键词： Zero-shot learning

Optimized Admittance Control for Manipulators Interacting with Unknown Environment

学校读者我要写书评

暂无评论

Optimized Admittance Control for Manipulators Interacting wi...

IEEE International Conference on Industrial technology (ICIT)

作者： Haiyi Kong Guangzhu Peng Guang Li Chenguang Yang Department of Electrical and Electronic Engineering University of Manchester Oxford Rd Manchester UK Bristol Robotics Laboratory University of the West of England Bristol UK School of Automation Nanjing University of Information Science and Technology Nanjing China Department of Computer Science University of Liverpool Liverpool UK

ISBN: (数字)9798350340266

ISBN: (纸本)9798350340273

This paper considers the study scenario that the end-effector of a manipulator follows a desired trajectory and interacts with external environment. To maximize the interaction performance, admittance control is combined with adaptive dynamic programming (ADP). The optimal admittance parameters can be learned online without prior knowledge of the environment. A data-driven Hybrid Iteration is employed in the ADP, which can relax the initial stabilizing requirement and at the same time has a faster convergence rate compared with Value Iteration. In addition, a more accurate environment model is considered in the system control design, where a general iterative expression is proposed to describe the varying contour of the environment. At last, simulation and experimental studies are given to verify the effectiveness of the proposed method.

关键词： Adaptation models Control design Heuristic algorithms End effectors Dynamic programming Trajectory Iterative methods