检索结果-内蒙古大学图书馆

Semantic Segmentation of Underwater Imagery: Dataset and Benchmark

学校读者我要写书评

暂无评论

Semantic Segmentation of Underwater Imagery: Dataset and Ben...

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Md Jahidul Islam Chelsey Edge Yuyang Xiao Peigen Luo Muntaqim Mehtaz Christopher Morse Sadman Sakib Enan Junaed Sattar Interactive Robotics and Vision Laboratory (IRVLab) Minnesota Robotics Institute (MnRI) University of Minnesota Twin Cities US

ISBN: (数字)9781728162126

ISBN: (纸本)9781728162133

In this paper, we present the first large-scale dataset for semantic Segmentation of Underwater IMagery (SUIM). It contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images have been rigorously collected during oceanic explorations and human-robot collaborative experiments, and annotated by human participants. We also present a comprehensive benchmark evaluation of several state-of-the-art semantic segmentation approaches based on standard performance metrics. Additionally, we present SUIM-Net, a fully-convolutional deep residual model that balances the trade-off between performance and computational efficiency. It offers competitive performance while ensuring fast end-to-end inference, which is essential for its use in the autonomy pipeline by visually-guided underwater robots. In particular, we demonstrate its usability benefits for visual servoing, saliency prediction, and detailed scene understanding. With a variety of use cases, the proposed model and benchmark dataset open up promising opportunities for future research in underwater robot vision.

关键词： Image segmentation Computational modeling Semantics Object segmentation Benchmark testing Unmanned underwater vehicles Fish

Underwater image super-resolution using deep residual multipliers

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Islam, Md Jahidul Enan, Sadman Sakib Luo, Peigen Sattar, Junaed Interactive Robotics and Vision Laboratory Department of Computer Science and Engineering Minnesota Robotics Institute University of Minnesota Twin Cities United States

We present a deep residual network-based generative model for single image super-resolution (SISR) of underwater imagery for use by autonomous underwater robots. We also provide an adversarial training pipeline for learning SISR from paired data. In order to supervise the training, we formulate an objective function that evaluates the perceptual quality of an image based on its global content, color, and local style information. Additionally, we present USR-248, a large-scale dataset of three sets of underwater images of ‘high’ (640 × 480) and ‘low’ (80 × 60, 160 × 120, and 320 × 240) resolution. USR-248 contains over 7K paired instances in each set of data for supervised training of 2×, 4×, or 8× SISR models. Furthermore, we validate the effectiveness of our proposed model through qualitative and quantitative experiments and compare the results with several state-of-the-art models’ performances. We also analyze its practical feasibility for applications such as scene understanding and attention modeling in noisy visual conditions. Copyright © 2019, The Authors. All rights reserved.

关键词： Large dataset

Robot-To-robot relative pose estimation using humans as markers

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Islam, Md Jahidul Mo, Jiawei Sattar, Junaed Interactive Robotics and Vision Laboratory Department of Computer Science and Engineering University of Minnesota Twin Cities United States

In this paper, we propose a method to determine the 3D relative pose of pairs of communicating robots by using human pose-based key-points as correspondences. We adopt a -leader-follower' framework, where at first, the leader robot visually detects and triangulates the key-points using the state-of-The-Art pose detector named OpenPose. Afterward, the follower robots match the corresponding 2D projections on their respective calibrated cameras and find their relative poses by solving the perspective-n-point (PnP) problem. In the proposed method, we design an efficient person re-identification technique for associating the mutually visible humans in the scene. Additionally, we present an iterative optimization algorithm to refine the associated key-points based on their local structural properties in the image space. We demonstrate that these refinement processes are essential to establish accurate key-point correspondences across viewpoints. Furthermore, we evaluate the performance of the proposed relative pose estimation method through several experiments conducted in terrestrial and underwater environments. Finally, we discuss the relevant operational challenges of this approach and analyze its feasibility for multi-robot cooperative systems in humandominated social settings and feature-deprived environments such as underwater. Copyright © 2019, The Authors. All rights reserved.

关键词： Iterative methods

Understanding Human Motion and Gestures for Underwater Human-Robot Collaboration

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Islam, Md Jahidul Interactive Robotics and Vision Laboratory Department of Computer Science and Engineering University of Minnesota- Twin Cities United States

In this paper, we present a number of robust methodologies for an underwater robot to visually detect, follow, and interact with a diver for collaborative task execution. We design and develop two autonomous diver-following algorithms, the first of which utilizes both spatial- and frequency-domain features pertaining to human swimming patterns in order to visually track a diver. The second algorithm uses a convolutional neural network-based model for robust tracking-by-detection. In addition, we propose a hand gesture-based human-robot communication framework that is syntactically simpler and computationally more efficient than the existing grammar-based frameworks. In the proposed interaction framework, deep visual detectors are used to provide accurate hand gesture recognition;subsequently, a finite-state machine performs robust and efficient gesture-to-instruction mapping. The distinguishing feature of this framework is that it can be easily adopted by divers for communicating with underwater robots without using artificial markers or requiring memorization of complex language rules. Furthermore, we validate the performance and effectiveness of the proposed methodologies through extensive field experiments in closed- and open-water environments. Finally, we perform a user interaction study to demonstrate the usability benefits of our proposed interaction framework compared to existing methods. Copyright © 2018, The Authors. All rights reserved.

关键词： Neural networks

Semantic segmentation of underwater imagery: Dataset and benchmark

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Islam, Md Jahidul Edge, Chelsey Xiao, Yuyang Luo, Peigen Mehtaz, Muntaqim Morse, Christopher Enan, Sadman Sakib Sattar, Junaed Interactive Robotics and Vision Laboratory Department of Computer Science and Engineering Minnesota Robotics Institute University of Minnesota Twin CitiesMN United States

In this paper, we present the first large-scale dataset for semantic Segmentation of Underwater IMagery (SUIM). It contains over 1500 images with pixel annotations for eight object categories: fish (vertebrates), reefs (invertebrates), aquatic plants, wrecks/ruins, human divers, robots, and sea-floor. The images are rigorously collected during oceanic explorations and human-robot collaborative experiments, and annotated by human participants. We also present a comprehensive benchmark evaluation of several state-of-the-art semantic segmentation approaches based on standard performance metrics. Additionally, we present SUIM-Net, a fully-convolutional deep residual model that balances the tradeoff between performance and computational efficiency. It offers competitive performance while ensuring fast end-to-end inference, which is essential for its use in the autonomy pipeline by visually-guided underwater robots. In particular, we demonstrate its usability benefits for visual servoing, saliency prediction, and detailed scene understanding. With a variety of use cases, the proposed model and benchmark dataset open up promising opportunities for future research on underwater robot vision. Copyright © 2020, The Authors. All rights reserved.

关键词： Semantic Segmentation

Simultaneous enhancement and super-resolution of underwater imagery for improved visual perception

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Islam, Md Jahidul Luo, Peigen Sattar, Junaed Interactive Robotics and Vision Laboratory Department of Computer Science and Engineering Minnesota Robotics Institute University of Minnesota Twin CitiesMN United States

In this paper, we introduce and tackle the simultaneous enhancement and super-resolution (SESR) problem for underwater robot vision and provide an efficient solution for near real-time applications. We present Deep SESR, a residual-in-residual network-based generative model that can learn to restore perceptual image qualities at 2×, 3×, or 4× higher spatial resolution. We supervise its training by formulating a multi-modal objective function that addresses the chrominance-specific underwater color degradation, lack of image sharpness, and loss in high-level feature representation. It is also supervised to learn salient foreground regions in the image, which in turn guides the network to learn global contrast enhancement. We design an end-to-end training pipeline to jointly learn the saliency prediction and SESR on a shared hierarchical feature space for fast inference. Moreover, we present UFO-120, the first dataset to facilitate large-scale SESR learning;it contains over 1500 training samples and a benchmark test set of 120 samples. By thorough experimental evaluation on the UFO-120 and other standard datasets, we demonstrate that Deep SESR outperforms the existing solutions for underwater image enhancement and super-resolution. We also validate its generalization performance on several test cases that include underwater images with diverse spectral and spatial degradation levels, and also terrestrial images with unseen natural objects. Lastly, we analyze its computational feasibility for single-board deployments and demonstrate its operational benefits for visuallyguided underwater robots. The model and dataset information will be available at: Https://***/ xahidbuffon/Deep-SESR. Copyright © 2020, The Authors. All rights reserved.

关键词： Optical resolving power

Fast underwater image enhancement for improved visual perception

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Islam, Md Jahidul Xia, Youya Sattar, Junaed Interactive Robotics and Vision Laboratory Department of Computer Science and Engineering Minnesota Robotics Institute University of Minnesota Twin CitiesMN United States

In this paper, we present a conditional generative adversarial network-based model for real-time underwater image enhancement. To supervise the adversarial training, we formulate an objective function that evaluates the perceptual image quality based on its global content, color, local texture, and style information. We also present EUVP, a largescale dataset of a paired and an unpaired collection of underwater images (of 'poor' and 'good' quality) that are captured using seven different cameras over various visibility conditions during oceanic explorations and human-robot collaborative experiments. In addition, we perform several qualitative and quantitative evaluations which suggest that the proposed model can learn to enhance underwater image quality from both paired and unpaired training. More importantly, the enhanced images provide improved performances of standard models for underwater object detection, human pose estimation, and saliency prediction. These results validate that it is suitable for real-time preprocessing in the autonomy pipeline by visually-guided underwater robots. The model and associated training pipelines are available at https://***/xahidbuffon/ funie-gan. Copyright © 2019, The Authors. All rights reserved.

关键词： Pipelines

SVAM: Saliency-guided Visual Attention Modeling by Autonomous Underwater Robots

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Islam, Md Jahidul Wang, Ruobing Sattar, Junaed Laboratory Dept. of ECE University of Florida FL United States Interactive Robotics and Vision Laboratory Dept. of CS University of Minnesota Twin Cities MN United States

This paper presents a holistic approach to saliency-guided visual attention modeling (SVAM) for use by autonomous underwater robots. Our proposed model, named SVAM-Net, integrates deep visual features at various scales and semantics for effective salient object detection (SOD) in natural underwater images. The SVAM-Net architecture is configured in a unique way to jointly accommodate bottom-up and top-down learning within two separate branches of the network while sharing the same encoding layers. We design dedicated spatial attention modules (SAMs) along these learning pathways to exploit the coarse-level and fine-level semantic features for SOD at four stages of abstractions. The bottom-up branch performs a rough yet reasonably accurate saliency estimation at a fast rate, whereas the deeper top-down branch incorporates a residual refinement module (RRM) that provides fine-grained localization of the salient objects. Extensive performance evaluation of SVAM-Net on benchmark datasets clearly demonstrates its effectiveness for underwater SOD. We also validate its generalization performance by several ocean trials' data that include test images of diverse underwater scenes and waterbodies, and also images with unseen natural objects. Moreover, we analyze its computational feasibility for robotic deployments and demonstrate its utility in several important use cases of visual attention modeling. Copyright © 2020, The Authors. All rights reserved.

关键词： Autonomous underwater vehicles

Underwater multi-robot convoying using visual tracking by detection

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Shkurti, Florian Chang, Wei-Di Henderson, Peter Islam, Md Jahidul Higuera, Juan Camilo Gamboa Li, Jimmy Manderson, Travis Xu, Anqi Dudek, Gregory Sattar, Junaed Centre for Intelligent Machines School of Computer Science McGill University Interactive Robotics and Vision Laboratory Department of Computer Science and Engineering University of Minnesota- Twin Cities Element AI

We present a robust multi-robot convoying approach that relies on visual detection of the leading agent, thus enabling target following in unstructured 3-D environments. Our method is based on the idea of tracking-by-detection, which interleaves efficient model-based object detection with temporal filtering of image-based bounding box estimation. This approach has the important advantage of mitigating tracking drift (i.e. drifting away from the target object), which is a common symptom of model-free trackers and is detrimental to sustained convoying in practice. To illustrate our solution, we collected extensive footage of an underwater robot in ocean settings, and hand-annotated its location in each frame. Based on this dataset, we present an empirical comparison of multiple tracker variants, including the use of several convolutional neural networks, both with and without recurrent connections, as well as frequency-based model-free trackers. We also demonstrate the practicality of this tracking-by-detection strategy in real-world scenarios by successfully controlling a legged underwater robot in five degrees of freedom to follow another robot’s independent motion. Copyright © 2017, The Authors. All rights reserved.

关键词： Multipurpose robots