检索结果-内蒙古大学图书馆

GraspGPT: Leveraging Semantic Knowledge from a Large Language Model for Task-Oriented Grasping

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Tang, Chao Huang, Dehao Ge, Wenqi Liu, Weiyu Zhang, Hong Shenzhen Key Laboratory of Robotics and Computer Vision Southern University of Science and Technology Shenzhen China Department of Electronic and Electrical Engineering Southern University of Science and Technology Shenzhen China Institute for Robotics and Intelligent Machines Georgia Institute of Technology Atlanta United States

Task-oriented grasping (TOG) refers to the problem of predicting grasps on an object that enable subsequent manipulation tasks. To model the complex relationships between objects, tasks, and grasps, existing methods incorporate semantic knowledge as priors into TOG pipelines. However, the existing semantic knowledge is typically constructed based on closed-world concept sets, restraining the generalization to novel concepts out of the pre-defined sets. To address this issue, we propose GraspGPT, a large language model (LLM) based TOG framework that leverages the open-end semantic knowledge from an LLM to achieve zero-shot generalization to novel concepts. We conduct experiments on Language Augmented TaskGrasp (LA-TaskGrasp) dataset and demonstrate that GraspGPT outperforms existing TOG methods on different held-out settings when generalizing to novel concepts out of the training set. The effectiveness of GraspGPT is further validated in real-robot experiments. Our code, data, appendix, and video are publicly available at https://***/view/graspgpt. Copyright © 2023, The Authors. All rights reserved.

关键词： Zero-shot learning

Boosting Adversarial Transferability across Model Genus by Deformation-Constrained Warping

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Qinliang Luo, Cheng Niu, Zenghao He, Xilin Xie, Weicheng Hou, Yuanbo Shen, Linlin Song, Siyang Computer Vision Institute School of Computer Science & Software Engineering Shenzhen University China Shenzhen Institute of Artificial Intelligence and Robotics for Society China Guangdong Key Laboratory of Intelligent Information Processing China WAVES Research Group Ghent University Belgium University of Leicester United Kingdom

Adversarial examples generated by a surrogate model typically exhibit limited transferability to unknown target systems. To address this problem, many transferability enhancement approaches (e.g., input transformation and model augmentation) have been proposed. However, they show poor performances in attacking systems having different model genera from the surrogate model. In this paper, we propose a novel and generic attacking strategy, called Deformation-Constrained Warping Attack (DeCoWA), that can be effectively applied to cross model genus attack. Specifically, DeCoWA firstly augments input examples via an elastic deformation, namely Deformation-Constrained Warping (DeCoW), to obtain rich local details of the augmented input. To avoid severe distortion of global semantics led by random deformation, DeCoW further constrains the strength and direction of the warping transformation by a novel adaptive control strategy. Extensive experiments demonstrate that the transferable examples crafted by our DeCoWA on CNN surrogates can significantly hinder the performance of Transformers (and vice versa) on various tasks, including image classification, video action recognition, and audio recognition. Code is made available at https://***/LinQinLiang/DeCoWA. © 2024, CC BY-NC-SA.

关键词： Semantics

Fast-moving object counting with an event camera

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Bialik, Kamil Kowalczyk, Marcin Blachut, Krzysztof Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Science and Technology Al. Mickiewicza 30 Krakow30-059 Poland

This paper proposes the use of an event camera as a component of a vision system that enables counting of fast-moving objects – in this case, falling corn grains. These type of cameras transmit information about the change in brightness of individual pixels and are characterised by low latency, no motion blur, correct operation in different lighting conditions, as well as very low power consumption. The proposed counting algorithm processes events in real time. The operation of the solution was demonstrated on a stand consisting of a chute with a vibrating feeder, which allowed the number of grains falling to be adjusted. The objective of the control system with a PID controller was to maintain a constant average number of falling objects. The proposed solution was subjected to a series of tests to determine the correctness of the developed method operation. On their basis, the validity of using an event camera to count small, fast-moving objects and the associated wide range of potential industrial applications can be confirmed. © 2022, CC BY.

关键词： Cameras

PointPillars Backbone Type Selection For Fast and Accurate LiDAR Object Detection

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2022年

作者： Lis, Konrad Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Science and Technology Al. Mickiewicza 30 Krakow30-059 Poland

3D object detection from LiDAR sensor data is an important topic in the context of autonomous cars and drones. In this paper, we present the results of experiments on the impact of backbone selection of a deep convolutional neural network on detection accuracy and computation speed. We chose the PointPillars network, which is characterised by a simple architecture, high speed, and modularity that allows for easy expansion. During the experiments, we paid particular attention to the change in detection efficiency (measured by the mAP metric) and the total number of multiply-addition operations needed to process one point cloud. We tested 10 different convolutional neural network architectures that are widely used in image-based detection problems. For a backbone like MobilenetV1, we obtained an almost 4x speedup at the cost of a 1.13% decrease in mAP. On the other hand, for CSPDarknet we got an acceleration of more than 1.5x at an increase in mAP of 0.33%. We have thus demonstrated that it is possible to significantly speed up a 3D object detector in LiDAR point clouds with a small decrease in detection efficiency. This result can be used when PointPillars or similar algorithms are implemented in embedded systems, including SoC FPGAs. The code is available at https://***/vision-agh/pointpillars backbone. © 2022, CC BY.

关键词： Object detection

Segment-Based Trajectory Prediction and Risk Assessment for RSU-assisted CAVs at Signalized Intersections

学校读者我要写书评

暂无评论

IEEE Transactions on Intelligent Vehicles

IEEE Transactions on Intelligent Vehicles 2024年 1-19页

作者： Cao, Yue Shangguan, Wei Visser, Arnoud Chen, Junjie Chai, Linguo Cai, Baigen School of Automation and Intelligence Beijing Jiaotong University Beijing China School of Automation and Intelligence and State Key Laboratory of Rail Traffic Control and Safety Beijing Jiaotong University Beijing China Intelligent Robotics and Computer Vision Lab of the Informatics Institute Faculty of Science University of Amsterdam The Netherlands

Detecting surrounding situations and reacting accordingly to avoid collisions remains a challenging task for autonomous driving. This task requires predicting the trajectories of surrounding agents and assessing the potential risk of future situations, which can be difficult to achieve solely through onboard vehicle devices. Therefore, this paper proposes a cooperative architecture for trajectory prediction and risk assessment conducted on roadside devices (RSUs) to assist Connected and Autonomous Vehicles (CAVs). Firstly, we develop a segmentbased prediction model (SegNet) tailored to hub signalized intersections. Intersections are divided into multiple segments, and the Curvilinear coordinates are utilized to indicate the geometric road features. The model leverages individual interaction cues in the ego segment and group features in the merging segments, while also incorporating traffic signal information to generate multimodal prediction results. In terms of risk assessment, we utilize the prediction results to provide hierarchical assistance, such as risk values, risk maps, and reference trajectories. Offline experimental results demonstrate that our SegNet model achieves competitive and well-balanced performance compared to stateof-the-art methods on the CitySim Database, with more accurate and smooth prediction trajectories. Through real-time CARLA and SUMO co-simulation, the performance of assisted CAVs indicates that they can safely and effectively navigate with the support of the proposed architecture. IEEE

关键词： Real time systems

Fast-moving object counting with an event camera

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2022年

关键词： Cameras

PointPillars Backbone Type Selection For Fast and Accurate LiDAR Object Detection

学校读者我要写书评

暂无评论

arXiv 2022年

关键词： Object detection

Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Zihan Song, Siyang Luo, Cheng Deng, Songhe Xie, Weicheng Shen, Linlin Computer Vision Institute School of Computer Science & Software Engineering Shenzhen University China Shenzhen Institute of Artificial Intelligence and Robotics for Society China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China University of Leicester United Kingdom Monash University Australia

Human facial action units (AUs) are mutually related in a hierarchical manner, as not only they are associated with each other in both spatial and temporal domains but also AUs located in the same/close facial regions show stronger relationships than those of different facial regions. While none of existing approach thoroughly model such hierarchical inter-dependencies among AUs, this paper proposes to comprehensively model multi-scale AU-related dynamic and hierarchical spatio-temporal relationship among AUs for their occurrences recognition. Specifically, we first propose a novel multi-scale temporal differencing network with an adaptive weighting block to explicitly capture facial dynamics across frames at different spatial scales, which specifically considers the heterogeneity of range and magnitude in different AUs’ activation. Then, a two-stage strategy is introduced to hierarchically model the relationship among AUs based on their spatial distribution (i.e., local and cross-region AU relationship modelling). Experimental results achieved on BP4D and DISFA show that our approach is the new state-of-the-art in the field of AU occurrence recognition. Our code is publicly available at https://***/CVI-SZU/MDHR. © 2024, CC BY.

关键词： Face recognition

Perspective Phase Angle Model for Polarimetric 3D Reconstruction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chen, Guangcheng He, Li Guan, Yisheng Zhang, Hong Guangdong University of Technology Guangzhou China Southern University of Science and Technology Shenzhen China Shenzhen Key Laboratory of Robotics and Computer Vision China

Current polarimetric 3D reconstruction methods, including those in the well-established shape from polarization literature, are all developed under the orthographic projection assumption. In the case of a large field of view, however, this assumption does not hold and may result in significant reconstruction errors in methods that make this assumption. To address this problem, we present the perspective phase angle (PPA) model that is applicable to perspective cameras. Compared with the orthographic model, the proposed PPA model accurately describes the relationship between polarization phase angle and surface normal under perspective projection. In addition, the PPA model makes it possible to estimate surface normals from only one single-view phase angle map and does not suffer from the so-called π-ambiguity problem. Experiments on real data show that the PPA model is more accurate for surface normal estimation with a perspective camera than the orthographic model. Copyright © 2022, The Authors. All rights reserved.

关键词： Image reconstruction