检索结果-内蒙古大学图书馆

Stability Analysis Framework for Particle-based Distance GANs with Wasserstein Gradient Flow

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Chuqi Wu, Yue Xiang, Yang Department of Mathematics Hong Kong University of Science and Technology Clear Water Bay Hong Kong Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Futian Shenzhen China

In this paper, we investigate the training process of generative networks that use a type of probability density distance named particle-based distance as the objective function, e.g. MMD GAN, Cramér GAN, EIEG GAN. However, these GANs often suffer from the problem of unstable training. In this paper, we analyze the stability of the training process of these GANs from the perspective of probability density dynamics. In our framework, we regard the discriminator D in these GANs as a feature transformation mapping that maps high dimensional data into a feature space, while the generator G maps random variables to samples that resemble real data in terms of feature space. This perspective enables us to perform stability analysis for the training of GANs using the Wasserstein gradient flow of the probability density function. We find that the training process of the discriminator is usually unstable due to the formulation of minG maxD E(G, D) in GANs. To address this issue, we add a stabilizing term in the discriminator loss function. We conduct experiments to validate our stability analysis and stabilizing method. Copyright © 2023, The Authors. All rights reserved.

关键词： Clustering algorithms

Learn Sharp Interface Solution by Homotopy Dynamics

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Chuqi Yang, Yahong Xiang, Yang Hao, Wenrui Department of Mathematics The Hong Kong University of Science and Technology Clear Water Bay Hong Kong Department of Mathematics The Pennsylvania State University State CollegePA United States Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Futian Shenzhen China

Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for sharp interface problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in sharp interface problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of sharp interface capturing. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving PDEs with sharp interfaces. © 2025, CC BY.

关键词： Contrastive learning

A fast neural hybrid Newton solver adapted to implicit methods for nonlinear dynamics

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jin, Tianyu Maierhofer, Georg Schratz, Katharina Xiang, Yang Department of Mathematics The Hong Kong University of Science and Technology Clear Water Bay Hong Kong Mathematical Institute University of Oxford United Kingdom Sorbonne Université France Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Futian Shenzhen China

The use of implicit time-stepping schemes for the numerical approximation of solutions to stiff nonlinear time-evolution equations brings well-known advantages including, typically, better stability behaviour and corresponding support of larger time steps, and better structure preservation properties. However, this comes at the price of having to solve a nonlinear equation at every time step of the numerical scheme. In this work, we propose a novel deep learning based hybrid Newton’s method to accelerate this solution of the nonlinear time step system for stiff time-evolution nonlinear equations. We propose a targeted learning strategy which facilitates robust unsupervised learning in an offline phase and provides a highly efficient initialisation for the Newton iteration leading to consistent acceleration of Newton’s method. A quantifiable rate of improvement in Newton’s method achieved by improved initialisation is provided and we analyse the upper bound of the generalisation error of our unsupervised learning strategy. These theoretical results are supported by extensive numerical results, demonstrating the efficiency of our proposed neural hybrid solver both in one- and two-dimensional cases. Copyright © 2024, The Authors. All rights reserved.

关键词： Nonlinear equations

Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equations

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Chuqi Yang, Yahong Xiang, Yang Hao, Wenrui Department of Mathematics The Hong Kong University of Science and Technology Clear Water Bay Hong Kong Hong Kong Department of Mathematics The Pennsylvania State University United States Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Shenzhen Futian China

Neural network-based approaches have recently shown significant promise in solving partial differential equations (PDEs) in science and engineering, especially in scenarios featuring complex domains or incorporation of empirical data. One advantage of the neural network methods for PDEs lies in its automatic differentiation (AD), which necessitates only the sample points themselves, unlike traditional finite difference (FD) approximations that require nearby local points to compute derivatives. In this paper, we quantitatively demonstrate the advantage of AD in training neural networks. The concept of truncated entropy is introduced to characterize the training property. Specifically, through comprehensive experimental and theoretical analyses conducted on random feature models and two-layer neural networks, we discover that the defined truncated entropy serves as a reliable metric for quantifying the residual loss of random feature models and the training speed of neural networks for both AD and FD methods. Our experimental and theoretical analyses demonstrate that, from a training perspective, AD outperforms FD in solving PDEs. Copyright © 2024, The Authors. All rights reserved.

关键词：

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG Representative learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fu, Xiaowen Wang, Bingxin Guo, Xinzhou Liu, Guoqing Xiang, Yang Department of Mathematics The Hong Kong University of Science and Technology Clear Water Bay Hong Kong Shenzhen Youjia Innov Tech Co. Ltd. Shenzhen China Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Futian Shenzhen China

Recently, multimodal electroencephalogram (EEG) learning has shown great promise in disease detection. At the same time, ensuring privacy in clinical studies has become increasingly crucial due to legal and ethical concerns. One widely adopted scheme for privacy protection is differential privacy (DP) because of its clear interpretation and ease of implementation. Although numerous methods have been proposed under DP, it has not been extensively studied for multimodal EEG data due to the complexities of models and signal data considered there. In this paper, we propose a novel Differentially Private Multimodal Laplacian Dropout (DP-MLD) scheme for multimodal EEG learning. Our approach proposes a novel multimodal representative learning model that processes EEG data by language models as text and other modal data by vision transformers as images, incorporating well-designed cross-attention mechanisms to effectively extract and integrate cross-modal features. To achieve DP, we design a novel adaptive feature-level Laplacian dropout scheme, where randomness allocation and performance are dynamically optimized within given privacy budgets. In the experiment on an open-source multimodal dataset of Freezing of Gait (FoG) in Parkinson’s Disease (PD), our proposed method demonstrates an approximate 4% improvement in classification accuracy, and achieves state-of-the-art performance in multimodal EEG learning under DP. Copyright © 2024, The Authors. All rights reserved.

关键词： Electroencephalography

Differentially Private Multimodal Laplacian Dropout (Dp-Mld) for Eeg Representative learning

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Fu, Xiaowen Wang, Bingxin Guo, Xinzhou Liu, Guoqing Xiang, Yang Department of Mathematics The Hong Kong University of Science and Technology Hong Kong Shenzhen Youjia Innov Tech Co. Ltd. Shenzhen China Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Shenzhen China Peng Cheng Laboratory China

Recently, multimodal electroencephalogram (EEG) learning has shown great promise in disease detection. At the same time, ensuring privacy in clinical studies has become increasingly crucial due to legal and ethical concerns. One widely adopted scheme for privacy protection is differential privacy (DP) because of its clear interpretation and ease of implementation. Although numerous methods have been proposed under DP, it has not been extensively studied for multimodal EEG data due to the complexities of models and signal data considered there. In this paper, we propose a novel Differentially Private Multimodal Laplacian Dropout (DP-MLD) scheme for multimodal EEG learning. Our approach proposes a novel multimodal representative learning model that processes EEG data by language models as text and other modal data by vision transformers as images, incorporating well-designed cross-attention mechanisms to effectively extract and integrate cross-modal features. To achieve DP, we design a novel adaptive feature-level Laplacian dropout scheme, where randomness allocation and performance are dynamically optimized within given privacy budgets. In the experiment on an open-source multimodal dataset of Freezing of Gait (FoG) in Parkinson's Disease (PD), our proposed method demonstrates an approximate 4\% improvement in classification accuracy, and achieves state-of-the-art performance in multimodal EEG learning under DP. © 2024, The Authors. All rights reserved.

关键词： Electroencephalography

ElasticLaneNet: An Efficient Geometry-Flexible Approach for Lane Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Feng, Yaxin Lan, Yuan Zhang, Luchan Xiang, Yang Department of Mathematics Hong Kong University of Science and Technology Clear Water Bay Hong Kong Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Futian Shenzhen China College of Mathematics and Statistics Shenzhen University Shenzhen518060 China

The task of lane detection involves identifying the boundaries of driving areas in real-time. Recognizing lanes with variable and complex geometric structures remains a challenge. In this paper, we explore a novel and flexible way of implicit lanes representation named Elastic Lane map (ELM), and introduce an efficient physics-informed end-to-end lane detection framework, namely, ElasticLaneNet (Elastic interaction energy-informed Lane detection Network). The approach considers predicted lanes as moving zero-contours on the flexibly shaped ELM that are attracted to the ground truth guided by an elastic interaction energy-loss function (EIE loss). Our framework well integrates the global information and low-level features. The method performs well in complex lane scenarios, including those with large curvature, weak geometry features at intersections, complicated cross lanes, Y-shapes lanes, dense lanes, etc. We apply our approach on three datasets: SDLane, CULane, and TuSimple. The results demonstrate exceptional performance of our method, with the state-of-the-art results on the structurally diverse SDLane, achieving F1-score of 89.51, Recall rate of 87.50, and Precision of 91.61 with fast inference speed. Copyright © 2023, The Authors. All rights reserved.

关键词： Geometry

Energy stable neural networks for gradient flow equations

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Yue Jin, Tianyu Chen, Chuqi Fan, Ganghua Lan, Yuan Zhang, Luchan Xiang, Yang Department of Mathematics The Hong Kong University of Science and Technology Clear Water Bay Hong Kong Algorithms of Machine Learning and Autonomous Driving Research Lab HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute Futian Shenzhen China School of Mathematical Sciences Shenzhen University Shenzhen518060 China

We propose an energy stable network (EStable-Net) for solving gradient flow equations. The EStable-Net enables decreasing of a discrete energy along the neural network, which is consistent with the property of the gradient flow equation. The architecture of the neural network EStable-Net is based on the block network structure (Autoflow) in which output of each block can be interpreted as an intermediate state of the evolution process of the equation, and the energy stable property is incorporated in each block, which is easily generalized to include other physical and/or numerical properties. Our EStable-Net is a supervised learning network approach for solving evolution equations which does not depend on the convergence of time step goes to 0, and can be applied generally even when only data is available but the equation is unknown. We also propose a training strategy for supervised learning that employs data of the evolution stages with different nature. The EStable-Net is validated by numerical experimental results based on the Allen-Cahn equation and the Cahn-Hilliard equation in two dimensions. Copyright © 2023, The Authors. All rights reserved.

关键词： Self-supervised learning

BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection

学校读者我要写书评

暂无评论

BEVHeight: A Robust Framework for Vision-based Roadside 3D O...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Lei Yang Kaicheng Yu Tao Tang Jun Li Kun Yuan Li Wang Xinyu Zhang Peng Chen State Key Laboratory of Automotive Safety and Energy Tsinghua University Autonomous Driving Lab Alibaba Group Shenzhen Campus Sun Yat-sen University Center for Machine Learning Research Peking University

While most recent autonomous driving system focuses on developing perception methods on ego-vehicle sensors, people tend to overlook an alternative approach to leverage intelligent roadside cameras to extend the perception ability beyond the visual range. We discover that the state-of-the-art vision-centric bird's eye view detection methods have inferior performances on roadside cameras. This is because these methods mainly focus on recovering the depth regarding the camera center, where the depth difference between the car and the ground quickly shrinks while the distance increases. In this paper, we propose a simple yet effective approach, dubbed BEVHeight, to address this issue. In essence, instead of predicting the pixel-wise depth, we regress the height to the ground to achieve a distance-agnostic formulation to ease the optimization process of camera-only perception methods. On popular 3D detection benchmarks of roadside cameras, our method surpasses all previous vision-centric methods by a significant margin. The code is available at https://***/ADlab-AutoDrive/BEVHeight.

关键词：