检索结果-内蒙古大学图书馆

Large-scale Pre-training for Grounded Video Caption Generation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Kazakos, Evangelos Schmid, Cordelia Sivic, Josef Czech Institute of Informatics Robotics and Cybernetics The Czech Technical University in Prague Czech Republic Inria École normale supérieure CNRS PSL Research University France

We propose a novel approach for captioning and object grounding in video, where the objects in the caption are grounded in the video via temporally dense bounding boxes. We introduce the following contributions. First, we present a large-scale automatic annotation method that aggregates captions grounded with bounding boxes across individual frames into temporally dense and consistent bounding box annotations. We apply this approach on the HowTo100M dataset to construct a large-scale pretraining dataset, named HowToGround1M. We also introduce a Grounded Video Caption Generation model, dubbed GROVE, and pre-train the model on HowToGround1M. Second, we introduce a new dataset, called iGround, of 3500 videos with manually annotated captions and dense spatio-temporally grounded bounding boxes. This allows us to measure progress on this challenging problem, as well as to fine-tune our model on this small-scale but high-quality data. Third, we demonstrate that our approach achieves state-ofthe-art results on the proposed iGround dataset compared to a number of baselines, as well as on the VidSTG and ActivityNet-Entities datasets. We perform extensive ablations that demonstrate the importance of pre-training using our automatically annotated HowToGround1M dataset followed by fine-tuning on the manually annotated iGround dataset and validate the key technical contributions of our model. © 2025, CC BY-NC-SA.

关键词： Large datasets

6D OBJECT POSE TRACKING IN INTERNET VIDEOS FOR ROBOTIC MANIPULATION

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Ponimatkin, Georgy Cífka, Martin Souček, Tomáš Fourmy, Médéric Labbé, Yann Petrík, Vladimír Sivic, Josef Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague Czech Republic Faculty of Electrical Engineering Czech Technical University in Prague Czech Republic H Company France

We seek to extract a temporally consistent 6D pose trajectory of a manipulated object from an Internet instructional video. This is a challenging set-up for current 6D pose estimation methods due to uncontrolled capturing conditions, subtle but dynamic object motions, and the fact that the exact mesh of the manipulated object is not known. To address these challenges, we present the following contributions. First, we develop a new method that estimates the 6D pose of any object in the input image without prior knowledge of the object itself. The method proceeds by (i) retrieving a CAD model similar to the depicted object from a large-scale model database, (ii) 6D aligning the retrieved CAD model with the input image, and (iii) grounding the absolute scale of the object with respect to the scene. Second, we extract smooth 6D object trajectories from Internet videos by carefully tracking the detected objects across video frames. The extracted object trajectories are then retargeted via trajectory optimization into the configuration space of a robotic manipulator. Third, we thoroughly evaluate and ablate our 6D pose estimation method on YCB-V and HOPE-Video datasets as well as a new dataset of instructional videos manually annotated with approximate 6D object trajectories. We demonstrate significant improvements over existing state-of-the-art RGB 6D pose estimation methods. Finally, we show that the 6D object motion estimated from Internet videos can be transferred to a 7-axis robotic manipulator both in a virtual simulator as well as in a real world set-up. We also successfully apply our method to egocentric videos taken from the EPIC-KITCHENS dataset, demonstrating potential for Embodied AI applications. © 2025, CC BY.

关键词： Manipulators

Is there anything left? Measuring semantic residuals of objects removed from 3D Gaussian Splatting

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Kocour, Simona Benbihi, Assia Adam, Aikaterini Sattler, Torsten Faculty of Electrical Engineering Czech Technical University in Prague Czech Republic Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague Czech Republic Archimedes Athena RC Greece

Searching in and editing 3D scenes has become extremely intuitive with trainable scene representations that allow linking human concepts to elements in the scene. These operations are often evaluated on the basis of how accurately the searched element is segmented or extracted from the scene. In this paper, we address the inverse problem, that is, how much of the searched element remains in the scene after it is removed. This question is particularly important in the context of privacy-preserving mapping when a user reconstructs a 3D scene and wants to remove private elements before sharing the map. To the best of our knowledge, this is the first work to address this question. To answer this, we propose a quantitative evaluation that measures whether a removal operation leaves object residuals that can be reasoned over. The scene is not private when such residuals are present. Experiments on state-of-the-art scene representations show that the proposed metrics are meaningful and consistent with the user study that we also present. We also propose a method to refine the removal based on spatial and semantic consistency. The evaluation framework and data are available at https://***/simonakocour/anything_left © 2025, CC BY-NC-SA.

关键词：

Fixing the Scale and Shift in Monocular Depth For Camera Pose Estimation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Ding, Yaqing Vávra, Václav Kocur, Viktor Yang, Jian Sattler, Torsten Kukelova, Zuzana Visual Recognition Group Faculty of Electrical Engineering Czech Technical University in Prague Czech Republic Faculty of Mathematics Physics and Informatics Comenius University in Bratislava Slovakia PCA Lab Nanjing University of Science and Technology Nanjing China Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague Czech Republic

Recent advances in monocular depth prediction have led to significantly improved depth prediction accuracy. In turn, this enables various applications to use such depth predictions. In this paper, we propose a novel framework for estimating the relative pose between two cameras from point correspondences with associated monocular depths. Since depth predictions are typically defined up to an unknown scale and shift parameter, our solvers jointly estimate both scale and shift parameters together with the camera pose. We derive efficient solvers for three cases: (1) two calibrated cameras, (2) two uncalibrated cameras with an unknown but shared focal length, and (3) two uncalibrated cameras with unknown and different focal lengths. Experiments on synthetic and real data, including experiments with depth maps estimated by 11 different depth predictors, show the practical viability of our solvers. Compared to prior work, our solvers achieve state-of-the-art results on two large-scale, real-world datasets. The source code is available at https://***/yaqding/pose_monodepth © 2025, CC BY.

关键词： Cameras

Visual Localization via Semantic Structures in Autonomous Photovoltaic Power Plant Inspection

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Kozák, Viktor Košnar, Karel Chudoba, Jan Kulich, Miroslav Přeučil, Libor Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague Jugoslávských partyzánů 1580/3 Prague 6160 00 Czech Republic Department of Cybernetics Faculty of Electrical Engineering Czech Technical University in Prague Karlovo náměstí 13 Prague 2121 35 Czech Republic

Inspection systems utilizing unmanned aerial vehicles (UAVs) equipped with thermal cameras are increasingly popular for the maintenance of photovoltaic (PV) power plants. However, automation of the inspection task is a challenging problem as it requires precise navigation to capture images from optimal distances and viewing angles. This paper presents a novel localization pipeline that directly integrates PV module detection with UAV navigation, allowing precise positioning during inspection. Detections are used to identify the power plant structures in the image and associate these with the power plant model. We define visually recognizable anchor points for the initial association and use object tracking to discern global associations. We present three distinct methods for visual segmentation of PV modules based on traditional computer vision, deep learning, and their fusion, and we evaluate their performance in relation to the proposed localization pipeline. The presented methods were verified and evaluated using custom aerial inspection data sets, demonstrating their robustness and applicability for real-time navigation. Additionally, we evaluate the influence of the power plant model’s precision on the localization methods. © 2025, CC BY.

关键词： Unmanned aerial vehicles (UAV)

A batch production scheduling problem in a reconfigurable hybrid manufacturing-remanufacturing system

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Vahedi-Nouri, Behdin Rohaninejad, Mohammad Hanzálek, Zdeněk Foumani, Mehdi School of Industrial Engineering College of Engineering University of Tehran Tehran Iran Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague Prague Czech Republic School of Intelligent Finance and Business Xi’an Jiaotong-Liverpool University Suzhou China

In recent years, remanufacturing of End-of-Life (EOL) products has been adopted by manufacturing sectors as a competent practice to enhance their sustainability, resiliency, and market share. Due to the mass customization of products and high volatility of market, processing of new products and remanufacturing of EOLs in a same shared facility, namely Hybrid Manufacturing-Remanufacturing System (HMRS), is a mean to keep such production efficient. Accordingly, customized production capabilities are required to increase flexibility, which can be suitably provided under the Reconfigurable Manufacturing System (RMS) paradigm. Despite the advantages of utilizing RMS technologies in HMRSs, production management of such systems suffers excessive complexity. Hence, this study concentrates on the production scheduling of an HMRS consisting of non-identical parallel reconfigurable machines where the orders can be grouped into batches. In this regard, Mixed-Integer Linear Programming (MILP) and Constraint Programming (CP) models are devised to formulate the problem. Furthermore, an efficient solution method is developed based on a Logic-based Benders Decomposition (LBBD) approach. The warm start technique is also implemented by providing a decent initial solution to the MILP model. Computational experiments attest to the LBBD method's superiority over the MILP, CP, and warm started MILP models by obtaining an average gap of about 2%, besides it provides valuable managerial insights. © 2025, CC BY-NC-ND.

关键词： Refining

PhysPose: Refining 6D Object Poses with Physical Constraints

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Malenickýn, Martin Cífka, Martin Fourmy, Médéric Montaut, Louis Carpentier, Justin Sivic, Josef Petrik, Vladimir Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague Czech Republic Faculty of Electrical Engineering Czech Technical University in Prague Czech Republic Inria - Département d’Informatique de l’École normale supérieure PSL Research University France

Accurate 6D object pose estimation from images is a key problem in object-centric scene understanding, enabling applications in robotics, augmented reality, and scene reconstruction. Despite recent advances, existing methods often produce physically inconsistent pose estimates, hindering their deployment in real-world scenarios. We introduce PhysPose, a novel approach that integrates physical reasoning into pose estimation through a postprocessing optimization enforcing non-penetration and gravitational constraints. By leveraging scene geometry, PhysPose refines pose estimates to ensure physical plausibility. Our approach achieves state-of-the-art accuracy on the YCB-Video dataset from the BOP benchmark and improves over the state-of-the-art pose estimation methods on the HOPE-Video dataset. Furthermore, we demonstrate its impact in robotics by significantly improving success rates in a challenging pick-and-place task, highlighting the importance of physical consistency in real-world applications. Copyright © 2025, The Authors. All rights reserved.

关键词： Geometry

Differentiable Simulation of Soft Robots with Frictional Contacts

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Ménager, Etienne Montaut, Louis Le Lidec, Quentin Carpentier, Justin Inria Département d’Informatique de l’École Normale Supérieure PSL Research University in Paris Paris75013 France Czech Institute of Informatics Robotics and Cybernetics Czech Technical University in Prague Praha16000 Czech Republic

In recent years, soft robotics simulators have evolved to offer various functionalities, including the simulation of different material types (e.g., elastic, hyper-elastic) and actuation methods (e.g., pneumatic, cable-driven, servomotor). These simulators also provide tools for various tasks, such as calibration, design, and control. However, efficiently and accurately computing derivatives within these simulators remains a challenge, particularly in the presence of physical contact interactions. Incorporating these derivatives can, for instance, significantly improve the convergence speed of control methods like reinforcement learning and trajectory optimization, enable gradient-based techniques for design, or facilitate end-to-end machine-learning approaches for model reduction. This paper addresses these challenges by introducing a unified method for computing the derivatives of mechanical equations within the finite element method framework, including contact interactions modeled as a nonlinear complementarity problem. The proposed approach handles both collision and friction phases, accounts for their nonsmooth dynamics, and leverages the sparsity introduced by mesh-based models. Its effectiveness is demonstrated through several examples of controlling and calibrating soft systems. Copyright © 2025, The Authors. All rights reserved.

关键词： Friction

Multi-Platform Teach-and-Repeat Navigation by Visual Place Recognition Based on Deep-Learned Local Features

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Truhlařík, Václav Pivoňka, Tomáš Kasarda, Michal Přeučil, Libor Czech Institute of Informatics Robotics and Cybernetics Czech technical University in Prague Jugoslávských partyzánů 1580/3 Praha 6160 00 Czech Republic Department of Cybernetics Faculty of Electrical Engineering Czech Technical University in Prague Karlovo náměstí 13 Praha 2121 35 Czech Republic

Uniform and variable environments still remain a challenge for stable visual localization and mapping in mobile robot navigation. One of the possible approaches suitable for such environments is appearance-based teach-and-repeat navigation, relying on simplified localization and reactive robot motion control - all without a need for standard mapping. This work brings an innovative solution to such a system based on visual place recognition techniques. Here, the major contributions stand in the employment of a new visual place recognition technique, a novel horizontal shift computation approach, and a multi-platform system design for applications across various types of mobile robots. Secondly, a new public dataset for experimental testing of appearance-based navigation methods is introduced. Moreover, the work also provides real-world experimental testing and performance comparison of the introduced navigation system against other state-of-the-art methods. The results confirm that the new system outperforms existing methods in several testing scenarios, is capable of operation indoors and outdoors, and exhibits robustness to day and night scene variations. © 2025, CC BY.

关键词： Mapping