检索结果-内蒙古大学图书馆

MetaEarth: A Generative Foundation Model for Global-scale Remote Sensing image Generation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Zhiping Liu, Chenyang Liu, Liqin Shi, Zhenwei Zou, Zhengxia The Image Processing Center School of Astronautics Beihang University Beijing100191 China The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China The Shanghai Artificial Intelligence Laboratory Shanghai200232 China Shen Yuan Honors College Beihang University Beijing100191 China The Department of Guidance Navigation and Control School of Astronautics Beihang University Beijing100191 China

The recent advancement of generative foundational models has ushered in a new era of image generation in the realm of natural images, revolutionizing art design, entertainment, environment simulation, and beyond. Despite producing high-quality samples, existing methods are constrained to generating images of scenes at a limited scale. In this paper, we present MetaEarth - a generative foundation model that breaks the barrier by scaling image generation to a global level, exploring the creation of worldwide, multi-resolution, unbounded, and virtually limitless remote sensing images. In MetaEarth, we propose a resolution-guided self-cascading generative framework, which enables the generating of images at any region with a wide range of geographical resolutions. To achieve unbounded and arbitrary-sized image generation, we design a novel noise sampling strategy for denoising diffusion models by analyzing the generation conditions and initial noise. To train MetaEarth, we construct a large dataset comprising multi-resolution optical remote sensing images with geographical information. Experiments have demonstrated the powerful capabilities of our method in generating global-scale images. Additionally, the MetaEarth serves as a data engine that can provide high-quality and rich training data for downstream tasks. Our model opens up new possibilities for constructing generative world models by simulating Earth’s visuals from an innovative overhead perspective. © 2024, CC BY.

关键词： Generative adversarial networks

IF2Net: Innately Forgetting-Free Networks for Continual Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Depeng Wang, Tianqi Xu, Bingrong Kawaguchi, Kenji Zeng, Zhigang Suganthan, Ponnuthurai Nagaratnam The School of Artificial Intelligence and Automation The Institute of Artificial Intelligence Huazhong University of Science and Technology The Key Laboratory of Image Processing and Intelligent Control Education Ministry of China Wuhan430074 China The School of Automation Wuhan University of Technology Wuhan430070 China The School of Computing National University of Singapore 117417 Singapore The School of Electrical and Electronic Engineering Nanyang Technological University Singapore639798 Singapore The KINDI Center for Computing Research College of Engineering Qatar University Doha Qatar

Continual learning can incrementally absorb new concepts without interfering with previously learned knowledge. Motivated by the characteristics of neural networks, in which information is stored in weights on connections, we investigated how to design an Innately Forgetting-Free Network (IF2Net) for continual learning context. This study proposed a straightforward yet effective learning paradigm by ingeniously keeping the weights relative to each seen task untouched before and after learning a new task. We first presented the novel representation-level learning on task sequences with random weights. This technique refers to tweaking the drifted representations caused by randomization back to their separate task-optimal working states, but the involved weights are frozen and reused (opposite to well-known layer-wise updates of weights). Then, sequential decision-making without forgetting can be achieved by projecting the output weight updates into the parsimonious orthogonal space, making the adaptations not disturb old knowledge while maintaining model plasticity. IF2Net allows a single network to inherently learn unlimited mapping rules without telling task identities at test time by integrating the respective strengths of randomization and orthogonalization. We validated the effectiveness of our approach in the extensive theoretical analysis and empirical study. Copyright © 2023, The Authors. All rights reserved.

关键词： Decision making

Real-time Multi-person Eyeblink Detection in the Wild for Untrimmed Video

学校读者我要写书评

暂无评论

Real-time Multi-person Eyeblink Detection in the Wild for Un...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Wenzheng Zeng Yang Xiao Sicheng Wei Jinfang Gan Xintao Zhang Zhiguo Cao Zhiwen Fang Joey Tianyi Zhou Key Laboratory of Image Processing and Intelligent Control Ministry of Education School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan China School of Biomedical Engineering Southern Medical University Guangzhou China Department of Rehabilitation Medicine Zhujiang Hospital Southern Medical University Guangzhou China Centre for Frontier AI Research Agency for Science Technology and Research (A*STAR) Singapore Institute of High Performance Computing Agency for Science Technology and Research (A*STAR) Singapore

Real-time eyeblink detection in the wild can widely serve for fatigue detection, face anti-spoofing, emotion analysis, etc. The existing research efforts generally focus on single-person cases towards trimmed video. However, multi-person scenario within untrimmed videos is also important for practical applications, which has not been well concerned yet. To address this, we shed light on this research field for the first time with essential contributions on dataset, theory, and practices. In particular, a large-scale dataset termed MPEblink that involves 686 untrimmed videos with 8748 eyeblink events is proposed under multi-person conditions. The samples are captured from uncon-strainedfilms to reveal “in the wild“ characteristics. Meanwhile, a real-time multi-person eyeblink detection method is also proposed. Being different from the existing counter-parts, our proposition runs in a one-stage spatio-temporal way with end-to-end learning capacity. Specifically, it simultaneously addresses the sub-tasks of face detection, face tracking, and human instance-level eyeblink detection. This paradigm holds 2 main advantages: (1) eyeblink features can be facilitated via the face's global context (e.g., head pose and illumination condition) with joint optimization and interaction, and (2) addressing these sub-tasks in parallel instead of sequential manner can save time remarkably to meet the real-time running requirement. Experiments on MPEblink verify the essential challenges of real-time multi-person eyeblink detection in the wild for untrimmed video. Our method also outperforms existing approaches by large margins and with a high inference speed.

关键词：

Adversarial Learning Network for Recovering Rppg Signals from Low-Resolution images for Remote Heart Rate Measurement

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Zhai, Dezhao Chen, Wei Li, Qinwei Yu, Ming Wu, Hang Tianjin Key Laboratory for Advanced Mechatronic System Design and Intelligent Control School of Mechanical Engineering Tianjin University of Technology Tianjin300384 China Systems Engineering Institute Academy of Military Sciences People's Liberation Army Tianjin300161 China Tianjin Key Laboratory for Advanced Signal Processing Civil Aviation University of China Tianjin300300 China School of Artificial Intelligence Nankai University Tianjin300381 China

Remote Photoplethysmography (rPPG) is a non-contact method for measuring heart rate (HR) through facial video, which has significant research value and application potential. However, in long-distance vital sign measurement tasks, the low resolution of facial images and insufficient pixels in the facial region severely degrade the quality of the rPPG signals, leading to a sharp decline in HR accuracy. To address this issue, this paper proposes an adversarial learning network for recovering rPPG signals from low-resolution images (ALRSL-HR). First, arbitrary resolution face images are fed into multi-channel network to extract rPPG signal features capturing rich rPPG information, generating rPPG feature blocks, and temporally aggregating for richer feature representation. These features are then inputted into the backbone network to generate adversarial examples of rPPG signals of varying qualities. Finally, a two-stage adversarial loss function is employed to enable the low-quality rPPG signals to learn the periodic characteristics of high-quality rPPG signals, recover the low-quality signals, and narrow the quality gap between rPPG signals from low-resolution images. Extensive experiments were conducted in this paper to validate the effectiveness of our model on the public COHFACE, UBFC-Phys and UBFC-rPPG datasets. The results of the experiments demonstrate that the proposed ALRSL-HR model can effectively recover degraded rPPG signals from low-resolution images. © 2024, The Authors. All rights reserved.

关键词： Photoplethysmography

Spatial non-locality induced non-markovian EIT in a single giant atom

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhu, Y.T. Wu, R.B. Xue, S. Department of Automation Shanghai Jiao Tong University Shanghai200240 China Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai200240 China Shanghai Engineering Research Center of Intelligent Control and Management Shanghai200240 China Department of Automation Tsinghua University Beijing100084 China Beijing National Research Center for Information Science and Technology Beijing100084 China

In recent experiments, electromagnetically induced transparency (EIT) were observed with giant atoms, but nothing unconventional were found from the transmission spectra. In this letter, we show that unconventional EIT does exist in giant atoms, and indicate why it has not been observed so far. Different from these existing works, this letter presents a consistent theory including a real space method and a time delayed master equation for observing unconventional EIT. We discover that this phenomenon is a quantum effect which cannot be correctly described in a semi-classical way as those in recent works. Our theory shows that it can be observed when the time delay between two neighboring coupling points is comparable to the relaxation time of the atom, which is crucial for a future experimental observation. This new phenomenon results from inherent non-locality of the giant atom, which physically forces propagating fields to be standing waves in space and the atom exhibiting retardations in time. Our theory establishes a framework for application of nonlocal systems to quantum information processing. Copyright © 2021, The Authors. All rights reserved.

关键词： Atoms

AnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yao, Yifei He, Wentao Gu, Chenyu Du, Jiaheng Tan, Fuwei Zhu, Zhen Lu, Junguo All authors are with Machine Vision and Autonomous System Laboratory Department of Automation School of Electrical Information and Electronic Engineering Shanghai Jiao Tong University Shanghai China Key Laboratory of System Control and Information Processing Ministry of Education of China China Shanghai Engineering Research Center of Intelligent Control and Management Shanghai200240 China University of Michigan Shanghai Jiao Tong University Joint Institute Shanghai Jiao Tong University Shanghai China

Training and deploying reinforcement learning (RL) policies for robots, especially in accomplishing specific tasks, presents substantial challenges. Recent advancements have explored diverse reward function designs, training techniques, simulation-to-reality (sim-to-real) transfers, and performance analysis methodologies, yet these still require significant human intervention. This paper introduces an end-to-end framework for training and deploying RL policies, guided by Large Language Models (LLMs), and evaluates its effectiveness on bipedal robots. The framework consists of three interconnected modules: an LLM-guided reward function design module, an RL training module leveraging prior work, and a sim-to-real homomorphic evaluation module. This design significantly reduces the need for human input by utilizing only essential simulation and deployment platforms, with the option to incorporate human-engineered strategies and historical data. We detail the construction of these modules, their advantages over traditional approaches, and demonstrate the framework’s capability to autonomously develop and refine controlling strategies for bipedal robot locomotion, showcasing its potential to operate independently of human intervention. Copyright © 2024, The Authors. All rights reserved.

关键词： Reinforcement learning

Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysis

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Chenyang Chen, keyan Zhang, Haotian Qi, Zipeng Zou, Zhengxia Shi, Zhenwei The Image Processing Center School of Astronautics Beihang University Beijing100191 China The Beijing Key Laboratory of Digital Media Beihang University Beijing100191 China The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China The Shanghai Artificial Intelligence Laboratory Shanghai200232 China Shen Yuan Honors College of Beihang University Beijing100191 China The Department of Guidance Navigation and Control School of Astronautics Beihang University Beijing100191 China Shanghai Artificial Intelligence Laboratory Shanghai200232 China

Monitoring changes in the Earth’s surface is crucial for understanding natural processes and human impacts, necessitating precise and comprehensive interpretation methodologies. Remote sensing satellite imagery offers a unique perspective for monitoring these changes, leading to the emergence of remote sensing image change interpretation (RSICI) as a significant research focus. Current RSICI technology encompasses change detection and change captioning, each with its limitations in providing comprehensive interpretation. To address this, we propose an interactive Change-Agent, which can follow user instructions to achieve comprehensive change interpretation and insightful analysis, such as change detection and change captioning, change object counting, change cause analysis, etc. The Change-Agent integrates a multi-level change interpretation (MCI) model as the eyes and a large language model (LLM) as the brain. The MCI model contains two branches of pixel-level change detection and semantic-level change captioning, in which the BI-temporal Iterative Interaction (BI3) layer is proposed to enhance the model’s discriminative feature representation capabilities. To support the training of the MCI model, we build the LEVIR-MCI dataset with a large number of change masks and captions of changes. Experiments demonstrate the SOTA performance of the MCI model in achieving both change detection and change description simultaneously, and highlight the promising application value of our Change-Agent in facilitating comprehensive interpretation of surface changes, which opens up a new avenue for intelligent remote sensing applications. © 2024, CC BY.

关键词： Change detection

Object Detection in 20 Years: A Survey

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zou, Zhengxia Chen, keyan Shi, Zhenwei Guo, Yuhong Ye, Jieping The Department of Guidance Navigation and Control School of Astronautics Beihang University Beijing100191 China The Shanghai Artificial Intelligence Laboratory Shanghai200232 China The Image Processing Center School of Astronautics The Beijing Key Laboratory of Digital Media China The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China The School of Computer Science Carleton University OttawaONK1S 5B6 Canada The Alibaba Group Hangzhou310030 China

Object detection, as of one the most fundamental and challenging problems in computer vision, has received great attention in recent years. Over the past two decades, we have seen a rapid technological evolution of object detection and its profound impact on the entire computer vision field. If we consider today's object detection technique as a revolution driven by deep learning, then back in the 1990s, we would see the ingenious thinking and long-term perspective design of early computer vision. This paper extensively reviews this fast-moving research field in the light of technical evolution, spanning over a quarter-century's time (from the 1990s to 2022). A number of topics have been covered in this paper, including the milestone detectors in history, detection datasets, metrics, fundamental building blocks of the detection system, speed-up techniques, and the recent state-of-the-art detection methods. Copyright © 2019, The Authors. All rights reserved.

关键词： Object detection

LDGCN: An Edge-End Lightweight Dual GCN Based on Single-Channel EEG for Driver Drowsiness Monitoring

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Jingwei Wang, Chuansheng Huang, Jiayan Fan, Haoyi Grau, Antoni Zhang, Fuquan The College of Computer and Big Data Fuzhou University Fuzhou China The Department of Automatic Control Polytechnic University of Catalonia Barcelona Spain The New Engineering Industry College Putian University Putian China School of Computer and Artificial Intelligence Zhengzhou University China Fujian Provincial Key Laboratory of Information Processing and Intelligent Control Minjiang University Fuzhou China College of Computer and Control Engineering Minjiang University Fuzhou China Key Laboratory of Sichuan Province Sichuan Conservatory of Music Chengdu China Fuzhou Technology Innovation Center of Intelligent Manufacturing information System Minjiang University Fuzhou China Fujian Polytechnic Normal University Fujian Province University Fuzhou China

Driver drowsiness electroencephalography (EEG) signal monitoring can timely alert drivers of their drowsiness status, thereby reducing the probability of traffic accidents. Graph convolutional networks (GCNs) have shown significant advancements in processing the non-stationary, time-varying, and non-Euclidean nature of EEG signals. However, the existing single-channel EEG adjacency graph construction process lacks interpretability, which hinders the ability of GCNs to effectively extract adjacency graph features, thus affecting the performance of drowsiness monitoring. To address this issue, we propose an edge-end lightweight dual graph convolutional network (LDGCN). Specifically, we are the first to incorporate neurophysiological knowledge to design a Baseline Drowsiness Status Adjacency Graph (BDSAG), which characterizes driver drowsiness status. Additionally, to express more features within limited EEG data, we introduce the Augmented Graph-level Module (AGM). This module captures global and local information at the graph level, ensuring that BDSAG features remain intact while enhancing effective feature expression capability. Furthermore, to deploy our method on the fourth-generation Raspberry Pi, we utilize Adaptive Pruning Optimization (APO) on both channels and neurons, reducing inference latency by almost half. Experiments on benchmark datasets demonstrate that LDGCN offers the best trade-off between monitoring performance and hardware resource utilization compared to existing state-of-the-art algorithms. All our source code can be found at https://***/BryantDom/Driver-Drowsiness-Monitoring. Copyright © 2024, The Authors. All rights reserved.

关键词： Electroencephalography