检索结果-内蒙古大学图书馆

LiDAR-based online 3D video object detection with graph-based message passing and spatiotemporal transformer attention

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yin, Junbo Shen, Jianbing Guan, Chenye Zhou, Dingfu Yang, Ruigang Beijing Lab of Intelligent Information Technology School of Computer Science Beijing Institute of Technology China Baidu Research National Engineering Laboratory of Deep Learning Technology and Application China Inception Institute of Artificial Intelligence United Arab Emirates University of Kentucky Kentucky United States

Existing LiDAR-based 3D object detectors usually focus on the single-frame detection, while ignoring the spatiotemporal information in consecutive point cloud frames. In this paper, we propose an end-to-end online 3D video object detector that operates on point cloud sequences. The proposed model comprises a spatial feature encoding component and a spatiotemporal feature aggregation component. In the former component, a novel Pillar Message Passing Network (PMPNet) is proposed to encode each discrete point cloud frame. It adaptively collects information for a pillar node from its neighbors by iterative message passing, which effectively enlarges the receptive field of the pillar feature. In the latter component, we propose an Attentive Spatiotemporal Transformer GRU (AST-GRU) to aggregate the spatiotemporal information, which enhances the conventional ConvGRU with an attentive memory gating mechanism. AST-GRU contains a Spatial Transformer Attention (STA) module and a Temporal Transformer Attention (TTA) module, which can emphasize the foreground objects and align the dynamic objects, respectively. Experimental results demonstrate that the proposed 3D video object detector achieves state-of-the-art performance on the large-scale nuScenes benchmark. Copyright © 2020, The Authors. All rights reserved.

关键词： Object detection

Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent.

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Hu, Wenqing Zhu, Zhanxing Xiong, Haoyi Huan, Jun Department of Mathematics and Statistics Missouri University of Science and Technology University of Missouri Rolla Peking University Beijing Institute of Big Data Research Beijing China Big Data Lab Baidu Inc. National Engineering Laboratory of Deep Learning Application and Technology

We interpret the variational inference of the Stochastic Gradient Descent (SGD) as minimizing a new potential function named the quasi-potential. We analytically construct the quasi-potential function in the case when the loss function is convex and admits only one global minimum point. We show in this case that the quasi-potential function is related to the noise covariance structure of SGD via a partial differential equation of Hamilton-Jacobi type. This relation helps us to show that anisotropic noise leads to faster escape than isotropic noise. We then consider the dynamics of SGD in the case when the loss function is non-convex and admits several different local minima. In this case, we demonstrate an example that shows how the noise covariance structure plays a role in "implicit regularization", a phenomenon in which SGD favors some particular local minimum points. This is done through the relation between the noise covariance structure and the quasi-potential function. Our analysis is based on Large Deviations Theory (LDT), and they are validated by numerical experiments. Copyright © 2019, The Authors. All rights reserved.

关键词： Stochastic systems

Invisible for both Camera and LiDAR: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Cao, Yulong Wang, Ningfei Xiao, Chaowei Yang, Dawei Fang, Jin Yang, Ruigang Chen, Qi Alfred Liu, Mingyan Li, Bo University of California Irvine United States University of Michigan United States NVIDIA Research Arizona State University Inceptio Baidu Research and National Engineering Laboratory of Deep Learning Technology and Application China University of Illinois at Urbana-Champaign

In Autonomous Driving (AD) systems, perception is both security and safety critical. Despite various prior studies on its security issues, all of them only consider attacks on camera- or LiDAR-based AD perception alone. However, production AD systems today predominantly adopt a Multi-Sensor Fusion (MSF) based design, which in principle can be more robust against these attacks under the assumption that not all fusion sources are (or can be) attacked at the same time. In this paper, we present the first study of security issues of MSF-based perception in AD systems. We directly challenge the basic MSF design assumption above by exploring the possibility of attacking all fusion sources simultaneously. This allows us for the first time to understand how much security guarantee MSF can fundamentally provide as a general defense strategy for AD perception. We formulate the attack as an optimization problem to generate a physically-realizable, adversarial 3D-printed object that misleads an AD system to fail in detecting it and thus crash into it. To systematically generate such a physical-world attack, we propose a novel attack pipeline that addresses two main design challenges: (1) non-differentiable target camera and LiDAR sensing systems, and (2) non-differentiable cell-level aggregated features popularly used in LiDAR-based AD perception. We evaluate our attack on MSF algorithms included in representative open-source industry-grade AD systems in real-world driving scenarios. Our results show that the attack achieves over 90% success rate across different object types and MSF algorithms. Our attack is also found stealthy, robust to victim positions, transferable across MSF algorithms, and physical-world realizable after being 3D-printed and captured by LiDAR and camera devices. To concretely assess the end-to-end safety impact, we further perform simulation evaluation and show that it can cause a 100% vehicle collision rate for an industry-grade AD system. We also evalu

关键词： Autonomous vehicles

Q-ViT: accurate and fully quantized low-bit vision transformer 22

学校读者我要写书评

暂无评论

Q-ViT: accurate and fully quantized low-bit vision transform...

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Yanjing Li Sheng Xu Baochang Zhang Xianbin Cao Peng Gao Guodong Guo Beihang University Beijing P.R.China Beihang University Beijing P.R.China and Zhongguancun Laboratory Beijing P.R.China Shanghai Artificial Intelligence Laboratory Shanghai P.R.China Institute of Deep Learning Baidu Research Beijing P.R.China and National Engineering Laboratory for Deep Learning Technology and Application Beijing P.R.China

ISBN: (纸本)9781713871088

The large pre-trained vision transformers (ViTs) have demonstrated remarkable performance on various visual tasks, but suffer from expensive computational and memory cost problems when deployed on resource-constrained devices. Among the powerful compression approaches, quantization extremely reduces the computation and memory consumption by low-bit parameters and bit-wise operations. However, low-bit ViTs remain largely unexplored and usually suffer from a significant performance drop compared with the real-valued counterparts. In this work, through extensive empirical analysis, we first identify the bottleneck for severe performance drop comes from the information distortion of the low-bit quantized self-attention map. We then develop an information rectification module (IRM) and a distribution guided distillation (DGD) scheme for fully quantized vision transformers (Q-ViT) to effectively eliminate such distortion, leading to a fully quantized ViTs. We evaluate our methods on popular DeiT and Swin backbones. Extensive experimental results show that our method achieves a much better performance than the prior arts. For example, our Q-ViT can theoretically accelerates the ViT-S by 6.14 × and achieves about 80.9% Top-1 accuracy, even surpassing the full-precision counterpart by 1.0% on ImageNet dataset.

关键词：

View extrapolation of human body from a single image

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Zhu, Hao Su, Hao Wang, Peng Cao, Xun Yang, Ruigang Nanjing University Nanjing China University of Kentucky LexingtonKY United States University of California San DiegoCA United States Baidu Inc. Beijing China National Engineering Laboratory of Deep Learning and Technology and Application China

We study how to synthesize novel views of human body from a single image. Though recent deep learning based methods work well for rigid objects, they often fail on objects with large articulation, like human bodies. The core step of existing methods is to fit a map from the observable views to novel views by CNNs;however, the rich articulation modes of human body make it rather challenging for CNNs to memorize and interpolate the data well. To address the problem, we propose a novel deep learning based pipeline that explicitly estimates and leverages the geometry of the underlying human body. Our new pipeline is a composition of a shape estimation network and an image generation network, and at the interface a perspective transformation is applied to generate a forward flow for pixel value transportation. Our design is able to factor out the space of data variation and makes learning at each step much easier. Empirically, we show that the performance for pose-varying objects can be improved dramatically. Our method can also be applied on real data captured by 3D sensors, and the flow generated by our methods can be used for generating high quality results in higher resolution. Copyright © 2018, The Authors. All rights reserved.

关键词： Pipelines

QuantYOLO: A High-Throughput and Power-Efficient Object Detection Network for Resource and Power Constrained UAVs

学校读者我要写书评

暂无评论

QuantYOLO: A High-Throughput and Power-Efficient Object Dete...

Proceedings of the Digital Image Computing: Technqiues and applications (DICTA)

作者： Muhammad Gohar Javed Minahil Raza Muhammad Mohsin Ghaffar Christian Weis Norbert Wehn Muhammad Shahzad Faisal Shafait School of Electrical Engineering and Computer Science National University of Sciences and Technology Pakistan Microelectronic Systems Design Research Group Technische Universität Kaiserslautern Germany Deep Learning Laboratory National Center of Artificial Intelligence (NCAI) Pakistan Data Science in Earth Observation TU Munich Germany

Convolutional Neural Networks (CNNs) are producing state-of-the-art results in the object detection field. However, deep topologies of CNN are computationally intensive and typically require excessive resources (i.e. high-end GPUs), which hinder their deployment on resource and power constrained UAVs. In this work, we present a high-throughput and power efficient quantized object detection network, QuantYOLO, which is based on the Tiny-YOLOv2 topology. We conduct a detailed exploration of precision and filter pruning vs. accuracy, throughput and power consumption trade-off for the object detection task. As a result of these explorations, we select a network with binarized weights and 4-bit activations (except the output layer), which is 21.8× smaller than the Tiny-YOLOv2 achieving a mean Average Precision (mAP) of 51.5% on the PASCAL-VOC dataset. Finally, we present an FPGA based accelerator, which achieves 1.6× higher throughput (FPS) and is 3.1× more power efficient as compared to prior FPGA architectures.

关键词： Quantization (signal) Power demand Network topology Object detection Computer architecture Throughput Topology

A unified object motion and affinity model for online Multi-Object Tracking

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yin, Junbo Wang, Wenguan Meng, Qinghao Yang, Ruigang Shen, Jianbing Beijing Lab of Intelligent Information Technology School of Computer Science Beijing Institute of Technology China ETH Zurich Switzerland Baidu Research China National Engineering Laboratory of Deep Learning Technology and Application China Inception Institute of Artificial Intelligence United Arab Emirates University of Kentucky Kentucky United States

Current popular online multi-object tracking (MOT) solutions apply single object trackers (SOTs) to capture object motions, while often requiring an extra affinity network to associate objects, especially for the occluded ones. This brings extra computational overhead due to repetitive feature extraction for SOT and affinity computation. Meanwhile, the model size of the sophisticated affinity network is usually non-trivial. In this paper, we propose a novel MOT framework that unifies object motion and affinity model into a single network, named UMA, in order to learn a compact feature that is discriminative for both object motion and affinity measure. In particular, UMA integrates single object tracking and metric learning into a unified triplet network by means of multi-task learning. Such design brings advantages of improved computation efficiency, low memory requirement and simplified training procedure. In addition, we equip our model with a task-specific attention module, which is used to boost task-aware feature learning. The proposed UMA can be easily trained end-to-end, and is elegant – requiring only one training stage. Experimental results show that it achieves promising performance on several MOT Challenge benchmarks. Copyright © 2020, The Authors. All rights reserved.

关键词： Benchmarking

Feature engineering Meets deep learning: A Case Study on Table Detection in Documents

学校读者我要写书评

暂无评论

Feature Engineering Meets Deep Learning: A Case Study on Tab...

Proceedings of the Digital Image Computing: Technqiues and applications (DICTA)

作者： Muhammad Ali Shahzad Rabeya Noor Sheraz Ahmad Ajmal Mian Faisal Shafait School of Electrical Engineering and Computer Science (SEECS) National University of Sciences and Technology (NUST) Islamabad Pakistan German Research Center for Artificial Intelligence (DFKI) Kaiserslautern Germany The University of Western Australia Perth Australia Deep Learning Laboratory National Center of Artificial Intelligence (NCAI) Islamabad Pakistan

Traditional computer vision approaches heavily relied on hand-crafted features for tasks such as visual object detection and recognition. The recent success of deep learning in automatically extracting representative and powerful features from images has brought a paradigm shift in this area. As a side effect, decades of research into hand-crafted features is considered outdated. In this paper, we present an approach for table detection in which we leverage a deep learning based table detection model with hand-crafted features from a classical table detection method. We demonstrate that by using a suitable encoding of hand-crafted features, the deep learning model is able to perform better at the detection task. Experiments on publicly available UNLV dataset show that the presented method achieves an accuracy comparable with the state-of-the-art deep learning methods without the need of extensive hyper-parameter tuning.

关键词： Feature extraction deep learning Image color analysis Object detection Task analysis Encoding Layout

deeply supervised depth map super-resolution as novel view synthesis

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Song, Xibin Dai, Yuchao Qin, Xueying School of Computer Science and Technology Shandong University China Baidu Research Beijing China National Engineering Laboratory of Deep Learning Technology and Application China Shaanxi Key Lab of Information Acquisition and Processing School of Electronics and Information Northwestern Polytechnical University School of Software Shandong University China

deep convolutional neural network (DCNN) has been successfully applied to depth map super-resolution and outperforms existing methods by a wide margin. However, there still exist two major issues with these DCNN based depth map super-resolution methods that hinder the performance: i) The low-resolution depth maps either need to be up-sampled before feeding into the network or substantial deconvolution has to be used;and ii) The supervision (high-resolution depth maps) is only applied at the end of the network, thus it is difficult to handle large up-sampling factors, such as ×8;×16. In this paper, we propose a new framework to tackle the above problems. First, we propose to represent the task of depth map super-resolution as a series of novel view synthesis sub-tasks. The novel view synthesis sub-task aims at generating (synthesizing) a depth map from different camera pose, which could be learned in parallel. Second, to handle large up-sampling factors, we present a deeply supervised network structure to enforce strong supervision in each stage of the network. Third, a multi-scale fusion strategy is proposed to effectively exploit the feature maps at different scales and handle the blocking effect. In this way, our proposed framework could deal with challenging depth map super-resolution efficiently under large up-sampling factors (e.g.×8;×16). Our method only uses the low-resolution depth map as input, and the support of color image is not needed, which greatly reduces the restriction of our method. Extensive experiments on various benchmarking datasets demonstrate the superiority of our method over current state-of-the-art depth map super-resolution methods. Copyright © 2018, The Authors. All rights reserved.

关键词： Optical resolving power