检索结果-内蒙古大学图书馆

Crowd Density Forecasting by Modeling Patch-Based Dynamics

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 287-294页

作者： Minoura, Hiroaki Yonetani, Ryo Nishimura, Mai Ushiku, Yoshitaka Chubu Univ Matsumotocho 1200 Kasugai Aichi 4878501 Japan OMRON SINIC X Corp Bunkyo Ku Hongo 5-24-5 Tokyo 1130033 Japan

Forecasting human activities observed in videos is a long-standing challenge in computer vision and robotics and is also beneficial for various real-world applications such as mobile robot navigation and drone landing. In this work, we present a new forecasting task called crowd density forecasting. Given a video of a crowd captured by a surveillance camera, our goal is to predict how the density of the crowd will change in unseen future frames. To address this task, we developed the patch-based density forecasting networks (PDFNs), which directly forecasts crowd density maps of future frames instead of trajectories of each moving person in the crowd. The PDFNs represent crowd density maps based on spatially or spatiotemporally overlapping patches and learn a simple density dynamics of fewer people in each patch. Doing so allows us to efficiently deal with diverse and complex crowd density dynamics observed when input videos involve a variable number of crowds moving independently. Experimental results with several public datasets of surveillance videos demonstrate the effectiveness of our approaches compared with state-of-the-art forecasting methods.

关键词： Forecasting Videos Trajectory Vehicle dynamics Surveillance Estimation Spatiotemporal phenomena Computer vision for transportation deep learning for visual perception surveillance robotic systems

来源：评论

学校读者我要写书评

暂无评论

PI-Net: An End-to-End deep Neural Network for Bidirectionally and Directly Fusing Point Clouds With Images

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 8647-8654页

作者： Wang, Qi Chen, Jian Deng, Jianqiang Zhang, Xinfang Zhejiang Univ State Key Lab Fluid Power & Mech Syst Coll Control Sci & Engn Hangzhou Peoples R China Zhejiang Univ Sch Mech Engn State Key Lab Fluid Power & Mechatron Syst Hangzhou Peoples R China

We present a novel network, PI-Net, for the fusion between point clouds and images in this letter. Most existing fusion methods project point clouds into pseudo images and then fuse the pseudo and RGB images with 2D CNNs. To get rid of structuring the pseudo images as the preprocessing, we propose a new fusion module, PI-fusion module. It can directly fuse points with pixels or grids which refer to the cells in the feature maps extracted from RGB images. Specifically, every point is aligned with a certain pixel/grid based on the pre-calibrated camera-LiDAR external parameters. The features aggregated from the points and pixels/grids are transferred to each other, which achieves the bidirectional fusion of the information. Then, we construct the PI-Net's fusion backbone by plugging the PI-fusion modules between a certain 2D CNN and a point-based network to fuse the features at every scale. The detection and the segmentation head networks further utilize the fused information to complete various tasks. The PI-Net unifies the point-cloud-to-image and image-to-point-cloud fusion into one network and achieves state-of-the-art results on the various tasks, such as the road detection, 3D object detection, and point cloud segmentation tasks on the KITTI benchmark.

关键词： deep learning for visual perception computer vision for transportation visual learning

来源：评论

学校读者我要写书评

暂无评论

Spatio-Temporal Convolutional Networks and N-Ary Ontologies for Human Activity-Aware Robotic System

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 620-627页

作者： Abdelkawy, H. Ayari, N. Chibani, A. Amirat, Y. Attal, F. Univ Paris Est Creteil UPEC LISSI Lab Cloud Robot F-77420 Champs Sur France ALTRAN Technol Res Dept F-78457 Velizy Villacoublay France

Endowing a companion robot with cognitive abilities to recognize human daily activities, in particular from body skeletons information, is a significant challenge, which needs complex and novel approaches. Recently, most of the proposed approaches exploit the hand-crafted features or the predefined traversal rules techniques to recognize human daily activities from skeleton information, which often lead to the deficit of robustness and generalization. In this work, a novel hybrid framework for human activity-aware robotic system is proposed. In the low-level, a novel Spatio-Temporal Joint based Convolutional Neural Network (STJ-CNN) is proposed to recognize human daily activities in the ambient environments. In the high-level, novel representation and inference services based on Narrative Knowledge Representation Language (NKRL) are proposed to represent and combine the detected human activities with the ambient events, and to infer the semantic context of the detected activity. Empirical experiments on real-world datasets have been conducted, besides an online demonstration created to validate the proposed approach. The final results show that the proposed approach outperforms the baseline models with a significant improvement up to 24% in terms of $F$-score on DAHLIA dataset.

关键词： Social human-robot interaction deep learning for visual perception recognition

来源：评论

学校读者我要写书评

暂无评论

Combining Events and Frames Using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 2822-2829页

作者： Gehrig, Daniel Rueegg, Michelle Gehrig, Mathias Hidalgo-Carrio, Javier Scaramuzza, Davide Univ Zurich Percept Grp CH-8050 Zurich Switzerland

Event cameras are novel vision sensors that report per-pixel brightness changes as a stream of asynchronous "events". They offer significant advantages compared to standard cameras due to their high temporal resolution, high dynamic range and lack of motion blur. However, events only measure the varying component of the visual signal, which limits their ability to encode scene context. By contrast, standard cameras measure absolute intensity frames, which capture a much richer representation of the scene. Both sensors are thus complementary. However, due to the asynchronous nature of events, combining them with synchronous images remains challenging, especially for learning-based methods. This is because traditional recurrent neural networks (RNNs) are not designed for asynchronous and irregular data from additional sensors. To address this challenge, we introduce Recurrent Asynchronous Multimodal (RAM) networks, which generalize traditional RNNs to handle asynchronous and irregular data from multiple sensors. Inspired by traditional RNNs, RAM networks maintain a hidden state that is updated asynchronously and can be queried at any time to generate a prediction. We apply this novel architecture to monocular depth estimation with events and frames where we show an improvement over state-of-the-art methods by up to 30% in terms of mean absolute depth error. To enable further research on multimodal learning with events, we release EventScape, a new dataset with events, intensity frames, semantic labels, and depth maps recorded in the CARLA simulator.

关键词： Data sets for robotic vision sensor fusion deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Learned Camera Gain and Exposure Control for Improved visual Feature Detection and Matching

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第2期6卷 2028-2035页

作者： Tomasi, Justin Wagstaff, Brandon Waslander, Steven L. Kelly, Jonathan Univ Toronto Inst Aerosp Studies UTIAS Space & Terr Autonomous Robot Syst STARS Lab Toronto ON M3H 5T6 Canada Univ Toronto Inst Aerosp Studies Toronto Robot & AILab TRAIL Toronto ON M3H 5T6 Canada

Successful visual navigation depends upon capturing images that contain sufficient useful information. In this letter, we explore a data-driven approach to account for environmental lighting changes, improving the quality of images for use in visual odometry (VO) or visual simultaneous localization and mapping (SLAM). We train a deep convolutional neural network model to predictively adjust camera gain and exposure time parameters such that consecutive images contain a maximal number of matchable features. The training process is fully self-supervised: our training signal is derived from an underlying VO or SLAM pipeline and, as a result, the model is optimized to perform well with that specific pipeline. We demonstrate through extensive real-world experiments that our network can anticipate and compensate for dramatic lighting changes (e.g., transitions into and out of road tunnels), maintaining a substantially higher number of inlier feature matches than competing camera parameter control algorithms.

关键词： deep learning for visual perception vision-based navigation visual learning

来源：评论

学校读者我要写书评

暂无评论

Weakly-Supervised Domain Adaptation of deep Regression Trackers via Reinforced Knowledge Distillation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第3期6卷 5016-5023页

作者： Dunnhofer, Matteo Martinel, Niki Micheloni, Christian Univ Udine Machine Learning & Percept Lab I-33100 Udine Italy

deep regression trackers are among the fastest tracking algorithms available, and therefore suitable for real-time robotic applications. However, their accuracy is inadequate in many domains due to distribution shift and overfitting. In this letter we overcome such limitations by presenting the first methodology for domain adaption of such a class of trackers. To reduce the labeling effort we propose a weakly-supervised adaptation strategy, in which reinforcement learning is used to express weak supervision as a scalar application-dependent and temporally-delayed feedback. At the same time, knowledge distillation is employed to guarantee learning stability and to compress and transfer knowledge from more powerful but slower trackers. Extensive experiments on five different robotic vision domains demonstrate the relevance of our methodology. Real-time speed is achieved on embedded devices and on machines without GPUs, while accuracy reaches significant results.

关键词： visual tracking computer vision for automation deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Multi-Frame Feature Aggregation for Real-Time Instrument Segmentation in Endoscopic Video

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 6773-6780页

作者： Lin, Shan Qin, Fangbo Peng, Haonan Bly, Randall A. Moe, Kris S. Hannaford, Blake Univ Washington Dept Elect & Comp Engn Seattle WA 98195 USA Chinese Acad Sci Res Ctr Precis Sensing & Control Inst Automat Beijing 100190 Peoples R China UW Dept Otolaryngol Head & Neck Surg Seattle WA 98105 USA

deep learning-based methods have achieved promising results on surgical instrument segmentation. However, the high computation cost may limit the application of deep models to time-sensitive tasks such as online surgical video analysis for robotic-assisted surgery. Moreover, current methods may still suffer from challenging conditions in surgical images such as various lighting conditions and the presence of blood. We propose a novel Multi-frame Feature Aggregation (MFFA) module to aggregate video frame features temporally and spatially in a recurrent mode. By distributing the computation load of deep feature extraction over sequential frames, we can use a lightweight encoder to reduce the computation costs at each time step. Moreover, public surgical videos usually are not labeled frame by frame, so we develop a method that can randomly synthesize a surgical frame sequence from a single labeled frame to assist network training. We demonstrate that our approach achieves superior performance to corresponding deeper segmentation models on two public surgery datasets.

关键词： Computer vision for medical robotics deep learning for visual perception object detection segmentation and categorization

来源：评论

学校读者我要写书评

暂无评论

ESPADA: Extended Synthetic and Photogrammetric Aerial-Image Dataset

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7981-7988页

作者： Lopez-Campos, Rafael Martinez-Carranza, Jose Inst Nacl Astrofis Opt & Electr Dept Comp Sci Puebla Mexico

We present a new aerial image dataset, named ES-PADA, intended for the training of deep neural networks for depth image estimation from a single aerial image. Given the difficulty of creating aerial image datasets containing image pairs of chromatic images related to their depth images, simulators such as AirSim have been proposed to generate synthetic images from photorealistic scenes. The latter enables the generation of thousands of images that can be used to train and evaluate neural models. However, we argue that synthetic photorealistic aerial image datasets can be improved by adding images generated from photogrammetric models imported into the simulator, thus enabling a less artificial generation of both chromatic and depth images. To assess the quality of these images, we compare the performance of 4 deep neural networks whose pre-trained models and code for re-training are publicly available. We also use ORB-SLAM, in its RGB-D version, to indirectly assess the estimated depth image. To accomplish this, chromatic images from 3 aerial videos and their depth images, estimated with the networks trained with ESPADA, are fed into ORB-SLAM. The estimated camera pose is compared against the trajectory retrieved from the GPS flight trajectory. Our results indicate that images generated from photogrammetric models improve the performance of depth estimation from a single aerial image.

关键词： Data sets for robotic vision deep learning for visual perception depth estimation aerial single image SLAM

来源：评论

学校读者我要写书评

暂无评论

ChangeGAN: A deep Network for Change Detection in Coarsely Registered Point Clouds

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 8277-8284页

作者： Nagy, Balazs Kovacs, Lorant Benedek, Csaba Eotvos Lorand Res Network Inst Comp Sci & Control SZ TAKI H-1083 Budapest Hungary Peter Pazmany Catholic Univ H-1083 Budapest Hungary

In this letter we introduce a novel change detection approach called ChangeGAN for coarsely registered point clouds in complex street-level urban environment. Our generative adversarial network-like (GAN) architecture compounds Siamese-style feature extraction, U-net-like use of multiscale features, and Spatial Trans-formation Network (STN) blocks for optimal transformation estimation. The input point clouds are represented by range images, which enables the use of 2D convolutional neural networks. The result is a pair of binary masks showing the change regions on each input range image, which can be backprojected to the input point clouds without loss of information. We have evaluated the proposed method on various challenging scenarios and we have shown its superiority against state-of-the-art change detection methods.

关键词： Change detection lidar deep learning for visual perception range sensing

来源：评论

学校读者我要写书评

暂无评论

ADAADepth: Adapting Data Augmentation and Attention for Self-Supervised Monocular Depth Estimation

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2021年第4期6卷 7791-7798页

作者： Kaushik, Vinay Jindgar, Kartik Lall, Brejesh IIT Delhi Dept Elect Engn Delhi 110016 India Manipal Univ Jaipur 302004 Rajasthan India

Self-supervised learning of depth has been a highly studied topic of research as it alleviates the requirement of having ground truth annotations for predicting depth. Depth is learnt as an intermediate solution to the task of view synthesis, utilising warped photometric consistency. Although it gives good results when trained using stereo data, the predicted depth is still sensitive to noise, illumination changes and specular reflections. Also, occlusion can be tackled better by learning depth from a single camera. We propose ADAA, utilising depth augmentation as depth supervision for learning accurate and robust depth. We propose a relational self-attention module that learns rich contextual features and further enhances depth results. We also optimize the auto-masking strategy across all losses by enforcing L1 regularisation over mask. Our novel progressive training strategy first learns depth at a lower resolution and then progresses to the original resolution with slight training. We utilise a ResNet18 encoder, learning features for prediction of both depth and pose. We evaluate our predicted depth on the standard KITTI driving dataset and achieve state-of-the-art results for monocular depth estimation whilst having significantly lower number of trainable parameters in our deep learning framework. We also evaluate our model on Make3D dataset showing better generalization than other methods.

关键词： deep learning for visual perception mapping SLAM

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：