检索结果-内蒙古大学图书馆

VG4D: Vision-Language Model Goes 4D Video Recognition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Deng, Zhichao Li, Xiangtai Li, Xia Tong, Yunhai Zhao, Shen Liu, Mengyuan School of Intelligent Systems Engineering Sun Yat-sen University Shenzhen China National Key Laboratory of General Artificial Intelligence Peking University China Department of Computer Science ETH Zurich Switzerland National Key Laboratory of General Artificial Intelligence Peking University Shenzhen Graduate School China PKU-Wuhan Institute for Artificial Intelligence China

Understanding the real world through point cloud video is a crucial aspect of robotics and autonomous driving systems. However, prevailing methods for 4D point cloud recognition have limitations due to sensor resolution, which leads to a lack of detailed information. Recent advances have shown that Vision-Language Models (VLM) pre-trained on web-scale text-image datasets can learn fine-grained visual concepts that can be transferred to various downstream tasks. However, effectively integrating VLM into the domain of 4D point clouds remains an unresolved problem. In this work, we propose the Vision-Language Models Goes 4D (VG4D) framework to transfer VLM knowledge from visual-text pretrained models to a 4D point cloud network. Our approach involves aligning the 4D encoder's representation with a VLM to learn a shared visual and text space from training on large-scale image-text pairs. By transferring the knowledge of the VLM to the 4D encoder and combining the VLM, our VG4D achieves improved recognition performance. To enhance the 4D encoder, we modernize the classic dynamic point cloud backbone and propose an improved version of PSTNet, im-PSTNet, which can efficiently model point cloud videos. Experiments demonstrate that our method achieves state-ofthe- art performance for action recognition on both the NTU RGB+D 60 dataset and the NTU RGB+D 120 dataset. Code is available at https://***/Shark0-0/VG4D. Copyright © 2024, The Authors. All rights reserved.

关键词： Visual languages

Many-Class Text Classification with Matching

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Song, Yi Gu, Yuxian Huang, Minlie The CoAI group Tsinghua University Beijing China Institute for Artificial Intelligence State Key Lab of Intelligent Technology and Systems Beijing National Research Center for Information Science and Technology Department of Computer Science and Technology Tsinghua University Beijing China

In this work, we formulate Text Classification as a Matching problem between the text and the labels, and propose a simple yet effective framework named TCM. Compared with previous text classification approaches, TCM takes advantage of the fine-grained semantic information of the classification labels, which helps distinguish each class better when the class number is large, especially in low-resource scenarios. TCM is also easy to implement and is compatible with various large pretrained language models. We evaluate TCM on 4 text classification datasets (each with 20+ labels) in both few-shot and full-data settings, and this model demonstrates significant improvements over other text classification paradigms. We also conduct extensive experiments with different variants of TCM and discuss the underlying factors of its success. Our method and analyses offer a new perspective on text classification. © 2022, CC BY.

关键词： Text processing

Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesis

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Kexin Lu, Jiamin Li, Junyou Yang, Xiaoran Du, Yuyang Wang, Kunyi Shi, Qiannuan Yu, Jiahui Li, Lanqing Qiu, Jiezhong Pan, Jianzhang Huang, Yi Fang, Qun Heng, Pheng Ann Chen, Guangyong Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong Institute of Intelligent Chemical Manufacturing and iChemFoundry Platform ZJU-Hangzhou Global Scientific and Technological Innovation Center Hangzhou China Institute of Microanalytical Systems Department of Chemistry Zhejiang University Hangzhou China Zhejiang Lab Hangzhou China Department of Information Engineering The Chinese University of Hong Kong Hong Kong

Recent AI research plots a promising future of automatic chemical reactions within the chemistry society. This study proposes Chemist-X, a comprehensive AI agent that automates the reaction condition optimization (RCO) task in chemical synthesis with retrieval-augmented generation (RAG) technology and AI-controlled wet-lab experiment executions. To begin with, as an emulation on how chemical experts solve the RCO task, Chemist-X utilizes a novel RAG scheme to interrogate available molecular and literature databases to narrow the searching space for later processing. The agent then leverages a computer-aided design (CAD) tool we have developed through a large language model (LLM) supervised programming interface. With updated chemical knowledge obtained via RAG, as well as the ability in using CAD tools, our agent significantly outperforms conventional RCO AIs confined to the fixed knowledge within its training data. Finally, Chemist-X interacts with the physical world through an automated robotic system, which can validate the suggested chemical reaction condition without human interventions. The control of the robotic system was achieved with a novel algorithm we have developed for the equipment, which relies on LLMs for reliable script generation. Results of our automatic wet-lab experiments, achieved by fully LLM-supervised end-to-end operation with no human in the lope, prove Chemist-X’s ability in self-driving laboratories. Copyright © 2023, The Authors. All rights reserved.

关键词： Biosynthesis

Speech-Vision Based Multi-Modal AI Control of a Magnetic Anchored and Actuated Endoscope

学校读者我要写书评

暂无评论

Speech-Vision Based Multi-Modal AI Control of a Magnetic Anc...

2022 IEEE International Conference on robotics and Biomimetics, ROBIO 2022

作者： Li, Jixiu Huang, Yisen Ng, Wing Yin Cheng, Truman Wu, Xixin Dou, Qi Meng, Helen Heng, Pheng Ann Liu, Yunhui Chan, Shannon Melissa Navarro-Alarcon, David Hang Ng, Calvin Sze Yan Chiu, Philip Wai Li, Zheng The Department of Surgery The Chinese University of Hong Kong Hong Kong Hong Kong The Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong Hong Kong Hong Kong The Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong Hong Kong The T Stone Robotics Institute The Department of Mechanical and Automation Engineering The Chinese University of Hong Kong Hong Kong Hong Kong The Department of Mechanical Engineering The Hong Kong Polytechnic University Hong Kong Hong Kong The Department of Surgery and the Chow Yuk Ho Technology Centre for Innovative Medicine The Chinese University of Hong Kong Hong Kong Hong Kong The Department of Surgery ChowYuk Ho Technology Centre of Innovative Medicine Li Ka Shing Institute of Health Science Multiscale Medical Robotics Center Ltd. The Chinese University of Hong Kong Hong Kong Hong Kong

ISBN: (纸本)9781665481090

In minimally invasive surgery (MIS), controlling the endoscope view is crucial for the operation. Many robotic endoscope holders were developed aiming to address this prob-lem,. These systems rely on joystick, foot pedal, simple voice command, etc. to control the robot. These methods requires surgeons extra effort and are not intuitive enough. In this paper, we propose a speech-vision based multi-modal AI approach, which integrates deep learning based instrument detection, automatic speech recognition and robot visual servo control. Surgeons could communicate with the endoscope by speech to indicate their view preference, such as the instrument to be tracked. The instrument is detected by the deep learning neural network. Then the endoscope takes the detected instrument as the target and follows it with the visual servo controller. This method is applied to a magnetic anchored and guided endoscope and evaluated experimentally. Preliminary results demonstrated this approach is effective and requires little efforts for the surgeon to control the endoscope view intuitively. © 2022 IEEE.

关键词： Endoscopy

Investigation of the Impact of Height and Bandwidth on Path Loss Exponent of 5G FR1 Wireless Networks at 800, 3500, and 5900 MHz 2

学校读者我要写书评

暂无评论

Investigation of the Impact of Height and Bandwidth on Path ...

2nd International Conference on Multidisciplinary Engineering and Applied science, ICMEAS 2023

作者： Imam-Fulani, Yusuf Olayinka Faruk, Nasir Usman, Aliyu D. Abdulkarim, Abubakar Tekanyi, Abdoulie M.S Yaro, Abdulmalik S. Alozie, Emmanuel Garba, Salisu Adewole, Kayode S. Oloyede, Abdulkarim A. Sowande, Olugbenga A. Abdullahi Baba, Bashir Musa, Abdulwaheed Imoize, Agbotiname Lucky Adediran, Yinusa A. Olagunju, Hawau I. Taura, Lawan S. University of Ilorin Department of Telecommunication Science Ilorin Nigeria Ahmadu Bello University Department of Electronics and Telecommunication Engineering Zaria Nigeria Sule Lamido University Department of Information Technology Kafin Hausa Nigeria Sule Lamido University Directorate of Information and Communication Technology Kafin Hausa Nigeria Ahmadu Bello University Department of Electrical Engineering Zaria Nigeria Kampala International University Department of Electrical and Telecommunication Engineering Kampala Uganda University of Hradec Králové Faculty of Informatics and Management Department of Informatics and Qualitative Methods Hradec Králové Czech Republic Sule Lamido University Department of Software Engineering Kafin Hausa Nigeria University of Ilorin Department of Computer Science Ilorin Nigeria Kwara State University Department of Electrical and Computer Engineering Malete Nigeria University of Johannesburg Institute for Intelligent Systems Johannesburg South Africa University of Lagos Department of Electrical and Electronics Engineering Lagos Nigeria Ruhr University Institute of Digital Communication Department of Electrical Engineering and Information Technology Bochum Germany University of Ilorin Department of Electrical and Electronics Engineering Ilorin Nigeria Sule Lamido University Department of Physics Kafin Hausa Nigeria

ISBN: (纸本)9798350358834

5G communication systems provide an end-to-end wireless connection to billions of users and devices across the globe. The quality of signals received during radio communication is notably influenced by the behavior of the radio propagation channel. A major parameter used in characterizing this channel is the Path Loss Exponent (PLE). Several works that have estimated and analyzed the PLE mostly considered the effect of distance and carrier frequency. However, the effect of base station antenna height and channel bandwidth on the PLE for 5G networks have not been adequately considered. To address this, the impact of antenna height of base station and channel bandwidth on the PLE within the 5G Frequency Range 1 (FR1) frequencies was investigated in this study, specifically at 800, 3500, and 5900 MHz. The licensed EDX Signal Pro software® with Cirrus high resolution global terrain and clutter data base was utilized to model, simulate and analyze the PLE for Kano City, Nigeria. Results showed that for the tested frequencies, an increase in either base station height or channel bandwidth leads to a significant reduction in the PLE. This study can be utilized by network planning engineers and the wireless research community to further improve network implementation and optimization toward understanding the behavior of signal propagation in 5G networks and beyond. © 2023 IEEE.

关键词： 5G mobile communication systems

Computational and Experimental Design of Fast and Versatile Magnetic Soft Robotic Low Re Swimmers

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Pramanik, Ratnadeep Park, Minjo Ren, Ziyu Sitti, Metin Verstappen, Roel W.C.P. Onck, Patrick R. Computational & Numerical Mathematics Group Bernoulli Institute for Mathematics Computer Science & Artificial Intelligence University of Groningen Netherlands Micromechanics Group Zernike Institute for Advanced Materials University of Groningen Netherlands Physical Intelligence Department Max Planck Institute for Intelligent Systems Stuttgart Germany School of Mechanical Engineering and Automation Beihang University Beijing China Institute for Biomedical Engineering ETH Zurich Switzerland School of Medicine College of Engineering Koc University Istanbul Turkey

Miniaturized magnetic soft robots have shown extraordinary capabilities of contactless manipulation, complex path maneuvering, precise localization, and rapid actuation, enabling them to cater to challenging biomedical applications such as targeted drug delivery, internal wound healing, and laparoscopic surgery. However, despite their successful fabrication by several different research groups, a thorough design strategy encompassing the optimized kinematic performance of the three fundamental biomimetic swimming modes at miniaturized length scales has not been reported until now. Here, we resolve this by designing magnetic soft robotic swimmers (MSRSs) from the class of helical and undulatory low Reynolds number (Re) swimmers using a fully coupled, experimentally calibrated computational fluid dynamics model. We study (and compare) their swimming performance, and report their steady-state swimming speed for different non-dimensional numbers that capture the competition by magnetic loading, nonlinear elastic deformation, and viscous solid-fluid coupling. We investigated their stability for different initial spatial orientations to ensure robustness during real-life applications. Our results show that the helical 'finger-shaped' swimmer is by far the fastest low Re swimmer in terms of body lengths per cycle, but that the undulatory 'carangiform-like' swimmer proved to be the most versatile, bidirectional swimmer with maximum stability. © 2024, The Authors. All rights reserved.

关键词： Reynolds number

Sleep Position Detection for Closed-Loop Treatment of Sleep-Related Breathing Disorders

学校读者我要写书评

暂无评论

Sleep Position Detection for Closed-Loop Treatment of Sleep-...

IEEE International Conference on Rehabilitation robotics (ICORR)

作者： A. Breuss N. Vonau C. Ungricht E. Schwarz M. Irion M. Bradicich F. A. Grewe S. Liechti S. Thiel M. Kohler R. Riener E. Wilhelm Department of Health Science and Technology Sensory-Motor Systems Lab Institute of Robotics and Intelligent Systems ETH Zurich Zurich Switzerland Sleep Disorders Centre of the Department of Pulmonology of the University Hospital Zurich University Hospital of Zurich (USZ) Zurich Switzerland Spinal Cord Injury Center University Hospital Balgrist Zurich Switzerland Engineering and Technology Institute Faculty of Science and Engineering University of Groningen The Netherlands

ISBN: (数字)9781665488297

ISBN: (纸本)9781665488303

Reliable detection of sleep positions is essential for the development of technical aids for patients with position-dependent sleep-related breathing disorders. We compare personalized and generalizable sleeping position classifiers using unobtrusive eight-channel pressure-sensing mats. Data of six male patients with confirmed position-dependent sleep apnea was recorded during three subsequent nights. Personalized position classifiers trained using leave-one-night-out cross-validation on average reached an F1-score of 61.3% for supine/non-supine and an F1-score of 46.2% for supine/lateral-left/lateral-right classification. The generalizable classifiers reached average F1-scores of 62.1% and 49.1% for supine/non-supine and supine/lateral-left/lateral-right classification, respectively. In-bed presence (“bed occupancy”) could be detected with an average F1-score of 98.1%. This work shows that personalized sleep-position classifiers trained with data from two nights have comparable performance to classifiers trained with large interpatient datasets. Simple eight-channel sensor mattresses can be used to accurately detect in-bed presence required for closed-loop systems but their use to classify sleep-positions is limited.

关键词： Pressure sensors Medical treatment Robot sensing systems Assistive robots Sleep apnea Robustness Real-time systems

Denoising Reuse: Exploiting Inter-frame Motion Consistency for Efficient Video Generation

学校读者我要写书评

暂无评论

IEEE Transactions on Circuits and systems for Video Technology 2025年

作者： Wang, Chenyu Yan, Shuo Chen, Yixuan Wang, Xianwei Wang, Yujiang Dong, Mingzhi Yang, Xiaochen Li, Dongsheng Zhu, Rui Clifton, David A. Dick, Robert P. Lv, Qin Yang, Fan Lu, Tun Gu, Ning Shang, Li Fudan University School of Computer Science Shanghai China Fudan University Shanghai Key Laboratory of Data Science Shanghai China Shanghai Institute of Intelligent Electronics & Systems Shanghai China Oxford Suzhou Centre for Advanced Research Suzhou China University of Bath Bath United Kingdom University of Glasgow School of Mathematics and Statistics United Kingdom Microsoft Research Asia Shanghai China University of London Bayes Business School City United Kingdom University of Oxford Department of Engineering Science Oxford United Kingdom University of Michigan Department of Electrical Engineering and Computer Science College of Engineering Ann ArborMI United States University of Colorado Boulder BoulderCO United States

Denoising-based diffusion models have attained impressive image synthesis;however, their applications on videos can lead to unaffordable computational costs due to the per-frame denoising operations. In pursuit of efficient video generation, we present a Diffusion Reuse MOtion (Dr. Mo) network to accelerate the video-based denoising process. Our crucial observation is that the latent representations in early denoising steps between adjacent video frames exhibit high consistencies with motion clues. Inspired by the discovery, we propose to accelerate the video denoising process by incorporating lightweight, learnable motion features. Specifically, Dr. Mo will only compute all denoising steps for base frames. For a non-based frame, Dr. Mo will propagate the pre-computed based latents of a particular step with interframe motions to obtain a fast estimation of its coarse-grained latent representation, from which the denoising will continue to obtain more sensitive and fine-grained representations. On top of this, Dr. Mo employs a meta-network named Denoising Step Selector (DSS) to dynamically determine the step to perform motion-based propagations for each frame, ensuring the correct transformation of multi-granularity visual features. Extensive evaluations on video generation and editing tasks indicate that Dr. Mo delivers widely applicable acceleration for diffusion-based video generations while effectively retaining the visual quality and style. Video generation and visualization results can be found at https://***. © 1991-2012 IEEE.

关键词： HTTP

Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Gu, Yuxian Ke, Pei Zhu, Xiaoyan Huang, Minlie The CoAI group Tsinghua University Beijing China Institute for Artificial Intelligence State Key Lab of Intelligent Technology and Systems Beijing National Research Center for Information Science and Technology Department of Computer Science and Technology Tsinghua University Beijing China

Training language models to learn from human instructions for zero-shot cross-task generalization has attracted much attention in NLP communities. Recently, instruction tuning (IT), which fine-tunes a pre-trained language model on a massive collection of tasks described via human-craft instructions, has been shown effective in instruction learning for unseen tasks. However, IT relies on a large amount of human-annotated samples, which restricts its generalization. Unlike labeled data, unlabeled data are often massive and cheap to obtain. In this work, we study how IT can be improved with unlabeled data. We first empirically explore the IT performance trends versus the number of labeled data, instructions, and training tasks. We find it critical to enlarge the number of training instructions, and the instructions can be underutilized due to the scarcity of labeled data. Then, we propose Unlabeled Data Augmented Instruction Tuning (UDIT) to take better advantage of the instructions during IT by constructing pseudo-labeled data from unlabeled plain texts. We conduct extensive experiments to show UDIT’s effectiveness in various scenarios of tasks and datasets. We also comprehensively analyze the key factors of UDIT to investigate how to better improve IT with unlabeled data. © 2022, CC BY.

关键词： Zero-shot learning