检索结果-内蒙古大学图书馆

VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wan, Zhaoliang Ling, Yonggen Yi, Senlin Qi, Lu Lee, Wangwei Lu, Minglei Yang, Sicheng Teng, Xiao Lu, Peng Yang, Xu Yang, Ming-Hsuan Cheng, Hui School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Robotics X Tencnet Shenzhen China The University of California Merced Merced United States Chinese Academic of Sciences Automation Institute Beijing China

This paper addresses the scarcity of large-scale datasets for accurate object-in-hand pose estimation, which is crucial for robotic in-hand manipulation within the "Perception-Planning-Control" paradigm. Specifically, we introduce VinT-6D, the first extensive multi-modal dataset integrating vision, touch, and proprioception, to enhance robotic manipulation. VinT-6D comprises 2 million VinT-Sim and 0.1 million VinT-Real splits, collected via simulations in MuJoCo and Blender and a custom-designed real-world platform. This dataset is tailored for robotic hands, offering models with whole-hand tactile perception and high-quality, well-aligned data. To the best of our knowledge, the VinT-Real is the largest considering the collection difficulties in the real-world environment so that it can bridge the gap of simulation to real compared to the previous works. Built upon VinT-6D, we present a benchmark method that shows significant improvements in performance by fusing multi-modal information. The project is available at https://***/. © 2024, CC BY.

关键词： Robot programming

Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Junxiong Wang, Yan Tao, Zeng Wang, Boyang Zhao, Qing Wang, Haorang Tong, Xuan Mai, Xinji Lin, Yuxuan Song, Wei Yu, Jiawen Yan, Shaoqi Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China East China University of Science and Technology Shanghai China College of Information Technology Shanghai Ocean University Shanghai China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

Pre-trained diffusion models utilized for image generation encapsulate a substantial reservoir of a priori knowledge pertaining to intricate textures. Harnessing the potential of leveraging this a priori knowledge in the context of image super-resolution presents a compelling avenue. Nonetheless, prevailing diffusion-based methodologies presently overlook the constraints imposed by degradation information on the diffusion process. Furthermore, these methods fail to consider the spatial variability inherent in the estimated blur kernel, stemming from factors such as motion jitter and out-of-focus elements in open-environment scenarios. This oversight results in a notable deviation of the image super-resolution effect from fundamental realities. To address these concerns, we introduce a framework known as Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution (SSR). Within the SSR framework, we propose a Spatially Variant Kernel Refinement (SVKR) module. SVKR estimates a Depth-Informed Kernel, which takes the depth information into account and is spatially variant. Additionally, SVKR enhance the accuracy of depth information acquired from LR images, allowing for mutual enhancement between the depth map and blur kernel estimates. Finally, we introduce the Adaptive Multi-Modal Fusion (AMF) module to align the information from three modalities: low-resolution images, depth maps, and blur kernels. This alignment can constrain the diffusion model to generate more authentic SR results. Copyright © 2024, The Authors. All rights reserved.

关键词： Image enhancement

SSD-MonoDETR: Supervised Scale-aware Deformable Transformer for Monocular 3D Object Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： He, Xuan Yang, Fan Yang, Kailun Lin, Jiacheng Fu, Haolong Wang, Meng Yuan, Jin Li, Zhiyong The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The School of Robotics Hunan University China410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The School of Computer Science Hefei University of Technology Hefei230009 China

Transformer-based methods have demonstrated superior performance for monocular 3D object detection recently, which aims at predicting 3D attributes from a single 2D image. Most existing transformer-based methods leverage both visual and depth representations to explore valuable query points on objects, and the quality of the learned query points has a great impact on detection accuracy. Unfortunately, existing unsupervised attention mechanisms in transformers are prone to generate low-quality query features due to inaccurate receptive fields, especially on hard objects. To tackle this problem, this paper proposes a novel "Supervised Scale-aware Deformable Attention" (SSDA) for monocular 3D object detection. Specifically, SSDA presets several masks with different scales and utilizes depth and visual features to adaptively learn a scale-aware filter for object query augmentation. Imposing the scale awareness, SSDA could well predict the accurate receptive field of an object query to support robust query feature generation. Aside from this, SSDA is assigned with a Weighted Scale Matching (WSM) loss to supervise scale prediction, which presents more confident results as compared to the unsupervised attention mechanisms. Extensive experiments on the KITTI and Waymo Open datasets demonstrate that SSDA significantly improves the detection accuracy, especially on moderate and hard objects, yielding state-of-the-art performance as compared to the existing approaches. Our code will be made publicly available at https://***/mikasa3lili/SSD-MonoDETR. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

RecGS: Removing Water Caustic with Recurrent Gaussian Splatting

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Tianyi Zhi, Weiming Huang, Kaining Mangelson, Joshua Barbalata, Corina Johnson-Roberson, Matthew Robotics Institute School of Computer Science Carnegie Mellon University PittsburghPA15213 United States Department of Mechanical & Industrial Engineering Louisiana State University Baton RougeLA70803 United States Department of Electrical and Computer Engineering Brigham Young University ProvoUT84602 United States

Water caustics are commonly observed in seafloor imaging data from shallow-water areas. Traditional methods that remove caustic patterns from images often rely on 2D filtering or pre-training on an annotated dataset, hindering the performance when generalizing to real-world seafloor data with 3D structures. In this paper, we present a novel method Recurrent Gaussian Splatting (RecGS), which takes advantage of today’s photorealistic 3D reconstruction technology, 3D Gaussian Splatting (3DGS), to separate caustics from seafloor imagery. With a sequence of images taken by an underwater robot, we build 3DGS recurrently and decompose the caustic with low-pass filtering in each iteration. In the experiments, we analyze and compare with different methods, including joint optimization, 2D filtering, and deep learning approaches. The results show that our proposed RecGS paradigm can effectively separate the caustic from the seafloor, improving the visual appearance, and can be potentially applied on more problems with inconsistent illumination. Our videos and code are released at https://***/proj/*** © 2024, CC BY.

关键词： Iterative methods

FDAM: Filter-Dedicated Approximate Multiplier Design for Real-Time CNN Acceleration

学校读者我要写书评

暂无评论

IEEE Transactions on computer-Aided Design of Integrated Circuits and Systems 2025年

作者： Nguyen, Kihwan Kim Xuan Truong Jeon, Dongsuk Lee, Hyuk-Jae Korea Republic of program Department of Electrical Engineering Seoul08826 Korea Republic of Inter-University Semiconductor Research Center Seoul National University Graduate School of Convergence Science and Technology Research Institute for Convergence Science Seoul08826 Korea Republic of Department of Electrical and Computer Engineering Seoul08826 Korea Republic of

Video super-resolution (VSR) is widely used in various high-definition applications, such as HDTVs and smartphones, requiring a dedicated upscaling technique for realtime full-HD generation. To reduce on-chip buffers for largesize output feature maps, a streaming VSR accelerator may employ an output stationary dataflow, leading to a large energy consumption caused by frequent filter switching. To mitigate this, we introduce a new filter-dedicated multiplier design for realtime VSR acceleration. We replace costly multipliers with adders, shifters, and multiplexers (MUXes), referred to as unified multiple constant multiplications (UMCM). The conventional UMCM, however, may incur considerable area/power overhead due to the unified topology constraint among different filter sets. To address this problem, we propose a new approximated MCM (AMCM) problem to relax the constraint and an approximate compatible graph synthesis (A-CGS) framework to efficiently solve AMCM by jointly searching for approximated filters and constructing a unified graph. Additionally, we suggest a lightweight finetuning method by freezing approximated filters and only finetuning biases, which can recover the original model’s accuracy within a few epochs. Experimental results with synthetic data demonstrate that AMCM reduces the area by up to 49.8%, 44.8%, and 40.3% when considering constant sets of 2, 4, and 8, respectively. Our designs with SR application achieve up to a 73.3% reduction in energy consumption. Experiments with the Set5 and Set14 datasets show that our model with bias correction achieves similar restoration performance compared to the eightbit models. © 1982-2012 IEEE.

关键词： Acceleration

A deep learning system for predicting time to progression of diabetic retinopathy

学校读者我要写书评

暂无评论

NATURE MEDICINE 2024年第2期30卷 358-359页

作者： [Anonymous] Shanghai Belt and Road International Joint Laboratory for Intelligent Prevention and Treatment of Metabolic Disorders Department of Computer Science and Engineering School of Electronic Information and Electrical Engineering Shanghai Jiao Tong University Department of Endocrinology and Metabolism Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine Shanghai Diabetes Institute Shanghai Clinical Center for Diabetes Shanghai China MOE Key Laboratory of AI School of Electronic Information and Electrical Engineering Shanghai Jiao Tong University Shanghai China Department of Ophthalmology Huadong Sanatorium Wuxi China Department of Ophthalmology Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine Shanghai China Department of Ophthalmology and Visual Sciences The Chinese University of Hong Kong Hong Kong China Singapore Eye Research Institute Singapore National Eye Centre Singapore Singapore Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong China Department of Chemical and Biological Engineering The Hong Kong University of Science and Technology Hong Kong China State Key Laboratory of Ophthalmology Zhongshan Ophthalmic Center Sun Yat-sen University Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science Guangzhou China Department of Ophthalmology Peking Union Medical College Hospital Peking Union Medical College Chinese Academy of Medical Sciences Beijing China Medical Records and Statistics Office Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine Shanghai China Department of Geriatrics Tongji Hospital Tongji Medical College Huazhong University of Science and Technology Wuhan China National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Tech

We developed and validated a deep learning system (termed DeepDR Plus) in a diverse, multiethnic, multi-country dataset to predict personalized risk and time to progression of diabetic retinopathy. We show that DeepDR Plus can be integrated into the clinical workflow to promote individualized intervention strategies for the management of diabetic retinopathy.

关键词： Diabetes complications Machine learning Predictive markers

STRAP: ROBOT SUB-TRAJECTORY RETRIEVAL FOR AUGMENTED POLICY LEARNING

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Memmel, Marius Berg, Jacob Chen, Bingqing Gupta, Abhishek Francis, Jonathan Paul G. Allen School of Computer Science & Engineering University of Washington United States Robot Learning Lab Bosch Center for Artificial Intelligence United States Robotics Institute Carnegie Mellon University United States

Robot learning is witnessing a significant increase in the size, diversity, and complexity of pre-collected datasets, mirroring trends in domains such as natural language processing and computer vision. Many robot learning methods treat such datasets as multi-task expert data and learn a multi-task, generalist policy by training broadly across them. Notably, while these generalist policies can improve the average performance across many tasks, the performance of generalist policies on any one task is often suboptimal due to negative transfer between partitions of the data, compared to task-specific specialist policies. In this work, we argue for the paradigm of training policies during deployment given the scenarios they encounter: rather than deploying pre-trained policies to unseen problems in a zero-shot manner, we non-parametrically retrieve and train models directly on relevant data at test time. Furthermore, we show that many robotics tasks share considerable amounts of low-level behaviors and that retrieval at the "sub"-trajectory granularity enables significantly improved data utilization, generalization, and robustness in adapting policies to novel problems. In contrast, existing full-trajectory retrieval methods tend to underutilize the data and miss out on shared cross-task content. This work proposes STRAP, a technique for leveraging pre-trained vision foundation models and dynamic time warping to retrieve sub-sequences of trajectories from large training corpora in a robust fashion. STRAP outperforms both prior retrieval algorithms and multi-task learning methods in simulated and real experiments, showing the ability to scale to much larger offline datasets in the real world as well as the ability to learn robust control policies with just a handful of real-world demonstrations. Project website at https://***/strap/. © 2024, CC BY.

关键词： Multi-task learning

A Portable Multiscopic Camera for Novel View and Time Synthesis in Dynamic Scenes

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Tianjia Lau, Yuen-Fui Chen, Qifeng The Hong Kong University of Science and Technology Hong Kong The Individualized Interdisciplinary Program The Robotics Institute Hong Kong The Department of Mathematics Hong Kong The Department of Computer Science and Engineering The Department of Electronic and Computer Engineering Hong Kong

We present a portable multiscopic camera system with a dedicated model for novel view and time synthesis in dynamic scenes. Our goal is to render high-quality images for a dynamic scene from any viewpoint at any time using our portable multiscopic camera. To achieve such novel view and time synthesis, we develop a physical multiscopic camera equipped with five cameras to train a neural radiance field (NeRF) in both time and spatial domains for dynamic scenes. Our model maps a 6D coordinate (3D spatial position, 1D temporal coordinate, and 2D viewing direction) to view-dependent and time-varying emitted radiance and volume density. Volume rendering is applied to render a photo-realistic image at a specified camera pose and time. To improve the robustness of our physical camera, we propose a camera parameter optimization module and a temporal frame interpolation module to promote information propagation across time. We conduct experiments on both real-world and synthetic datasets to evaluate our system, and the results show that our approach outperforms alternative solutions qualitatively and quantitatively. Our code and dataset are available at https://***/. Copyright © 2022, The Authors. All rights reserved.

关键词： Cameras

Analog Magnetic Sensor-Robotic System for Steel Structure Inspection

学校读者我要写书评

暂无评论

Analog Magnetic Sensor-Robotic System for Steel Structure In...

IEEE/SICE International Symposium on System Integration

作者： Anh Quyen Pham Duc La Hung Manh La AIR-VIET Corporation in Vietnam Kahlert School of Computing University of Utah Salt Lake City UT USA Department of Computer Science and Engineering Advanced Robotics and Automation (ARA) Lab University of Nevada Reno NV USA

More and more steel bridge collapse accidents occur worldwide due to broken steel structures, causing significant l oss o f l ife a nd p roperty t o m ankind. T his has spurred much research into robots that can climb steel surfaces and carry smart sensors with the desire to assist inspectors in inspecting steel defects. However, current non-destructive evaluation (NDE) sensors such as eddy-current and giant magnetoresistance (GMR) are not able to be integrated with the robotic platforms due to their large size, mass, and limited ability to interface. More importantly, these NDE sensors may fail to detect hidden and underlying cracks. Therefore, in this paper, we present a novel and compact analog magnetic sensor system to detect different types of cracks in steel. Validations are carried out on a steel test plate 600mm long, 140mm wide, and 6mm thick, and precision machined by CNCs to produce various man-made cracks including penetrating cracks, surface cracks, internal/hidden cracks, and underlying cracks. These cracks have a width varying from 0.6mm to 1.2mm with different depth levels. Combined with applying the Kalman filter t o t he n oise r eduction s ystem, t he r esults o btained are accurate, and the response speed is fast. The sensor is small enough and has firmware/software with an interface to make it possible to be integrated with a small drone or a climbing robot to perform an in-depth inspection of the steel bridge. A demonstration of the system can be seen in this video: https://***/OrdpLdnFlYk

关键词：