The simplified general perturbations 4 (SGP4) orbital propagation model is one of the most widely used methods for rapidly and reliably predicting the positions and velocities of objects orbiting Earth. Over time, SGP...
详细信息
The simplified general perturbations 4 (SGP4) orbital propagation model is one of the most widely used methods for rapidly and reliably predicting the positions and velocities of objects orbiting Earth. Over time, SGP models have undergone refinement to enhance their efficiency and accuracy. Nevertheless, they still do not match the precision offered by high-precision numerical propagators, which can predict the positions and velocities of space objects in low-Earth orbit with significantly smaller errors. In this study, we introduce a novel differentiable version of SGP4, named d SGP4. By porting the source code of SGP4 into a differentiable program based on PyTorch, we unlock a whole new class of techniques enabled by differentiable orbit propagation, including spacecraft orbit determination, state conversion, covariance similarity transformation, state transition matrix computation, and covariance propagation. Besides differentiability, our d SGP4 supports parallel propagation of a batch of two-line elements (TLEs) in a single execution and it can harness modern hardware accelerators like GPUs or XLA devices (e.g. TPUs) thanks to running on the PyTorch backend. Furthermore, the design of d SGP4 makes it possible to use it as a differentiable component in larger machine learning (ML) pipelines, where the propagator can be an element of a larger neural network that is trained or fine-tuned with data. Consequently, we propose a novel orbital propagation paradigm, ML-d SGP4. In this paradigm, the orbital propagator is enhanced with neural networks attached to its input and output. Through gradient-based optimization, the parameters of this combined model can be iteratively refined to achieve precision surpassing that of SGP4. Fundamentally, the neural networks function as identity operators when the propagator adheres to its default behavior as defined by SGP4. However, owing to the differentiability ingrained within d SGP4, the model can be fine-tuned with ephemeris
We have developed a differentiable programming framework for truncated hierarchical B-splines (THB-splines), which can be used for several applications in geometry modeling, such as surface fitting and deformable imag...
详细信息
We have developed a differentiable programming framework for truncated hierarchical B-splines (THB-splines), which can be used for several applications in geometry modeling, such as surface fitting and deformable image registration, and can be easily integrated with geometric deep learning frameworks. differentiable programming is a novel paradigm that enables an algorithm to be differentiated via automatic differentiation, i.e., using automatic differentiation to compute the derivatives of its outputs with respect to its inputs or parameters. differentiable programming has been used extensively in machine learning for obtaining gradients required in optimization algorithms such as stochastic gradient descent (SGD). While incorporating differentiable programming with traditional functions is straightforward, it is challenging when the functions are complex, such as splines. In this work, we extend the differentiable programming paradigm to THB-splines. THB-splines offer an efficient approach for complex surface fitting by utilizing a hierarchical tensor structure of B-splines, enabling local adaptive refinement. However, this approach brings challenges, such as a larger computational overhead and the non-trivial implementation of automatic differentiation and parallel evaluation algorithms. We use custom kernel functions for GPU acceleration in forward and backward evaluation that are necessary for differentiable programming of THB-splines. Our approach not only improves computational efficiency but also significantly enhances the speed of surface evaluation compared to previous methods. Our differentiable THB-splines framework facilitates faster and more accurate surface modeling with local refinement, with several applications in CAD and isogeometric analysis.
While deep learning and data -driven modeling approaches based on deep neural networks (DNNs) have recently attracted increasing attention for solving partial differential equations, their practical applications to re...
详细信息
While deep learning and data -driven modeling approaches based on deep neural networks (DNNs) have recently attracted increasing attention for solving partial differential equations, their practical applications to real -world scientific and engineering problems remain limited due to the relatively low accuracy and high computational costs. In this study, we present a differentiable programming -based hybrid meshfree approach within the field of computational mechanics. This approach seamlessly integrates neural network -based deep learning architectures with physics -based meshfree discretization techniques, thus referred to as the neural -integrated meshfree (NIM) method. To effectively approximate the solution, NIM employs a hybrid scheme called NeuroPU approximation, which combines continuous DNN representations with partition of unity (PU) basis functions associated with the underlying spatial discretization. This neural -numerical hybridization not only enhances the solution representation through functional space decomposition but also reduces both the size of DNN model and the need of automatic differentiation for spatial gradient computations, leading to a significant improvement in training efficiency. Under the NIM framework, we propose two truly meshfree solvers: the strong form -based NIM (S-NIM) and the local variational form -based NIM (V-NIM). In the S-NIM solver, the strong -form governing equation is directly considered in the loss function, while the V-NIM solver employs a local Petrov-Galerkin approach that allows the construction of variational residuals based on arbitrary overlapping subdomains. This ensures both the satisfaction of underlying physics and the preservation of the meshfree property. We perform extensive numerical experiments on both stationary and transient benchmark problems to assess the effectiveness of the proposed NIM methods in terms of accuracy, scalability, generalizability, and convergence properties. Moreover, comparati
differentiable programming is an emerging programming paradigm that allows people to take derivative of an output of arbitrary code snippet with respect to its input. It is the workhorse behind several well known deep...
详细信息
differentiable programming is an emerging programming paradigm that allows people to take derivative of an output of arbitrary code snippet with respect to its input. It is the workhorse behind several well known deep learning frameworks,and has attracted significant attention in scientific machine learning community. In this paper, we introduce and implement a density matrix based Hartree–Fock method that naturally fits into the demands of this paradigm, and demonstrate it by performing fully variational ground state calculation on several representative chemical molecules.
Mapping the functional connectome has the potential to uncover key insights into brain organisation. However, existing workflows for functional connectomics are limited in their adaptability to new data, and principle...
详细信息
Mapping the functional connectome has the potential to uncover key insights into brain organisation. However, existing workflows for functional connectomics are limited in their adaptability to new data, and principled workflow design is a challenging combinatorial problem. We introduce an analytic paradigm that implements common operations used in functional connectomics as fully differentiable processing blocks. Under this paradigm, workflow configurations exist as reparameterisations of a differentiable functional that interpolates them. The differentiable program that we ultimately envision occupies a niche midway between traditional pipelines and end-to-end neural networks, combining the glass-box tractability and domain knowledge of the former with the amenability to optimisation of the latter. In this preliminary work, we provide a proof of concept for differentiable connectomics, demonstrating the capacity of our processing blocks across three separate problem domains critically important to brain mapping. We also provide a software library to facilitate adoption. Our differentiable framework is competitive with state-of-the-art methods in functional brain parcellation, time series denoising, and covariance modelling. Taken together, our results demonstrate the promise of differentiable programming for functional connectomics.
The notion of a Moreau envelope is central to the analysis of first-order optimization algorithms for machine learning and signal processing. We define a compositional calculus adapted to Moreau envelopes and show how...
详细信息
ISBN:
(纸本)9781665405409
The notion of a Moreau envelope is central to the analysis of first-order optimization algorithms for machine learning and signal processing. We define a compositional calculus adapted to Moreau envelopes and show how to apply it to deep networks, and, more broadly, to learning systems equipped with automatic differentiation and implemented in the spirit of differentiable programming.
Automatic differentiation (AD) is a central algorithm in deep learning and the emerging field of differentiable programming. However, the performance of AD remains a significant bottleneck in these fields. Training la...
详细信息
ISBN:
(纸本)9798400700880
Automatic differentiation (AD) is a central algorithm in deep learning and the emerging field of differentiable programming. However, the performance of AD remains a significant bottleneck in these fields. Training large models requires repeatedly evaluating gradients via AD potentially millions of times. Additionally, the most common form of AD incurs an asymptotically large memory cost relative to the original function being differentiated. This paper introduces LAGrad, a reverse-mode, source-to-source AD system that leverages high-level information in MLIR to produce efficient differentiated code. LAGrad employs a collection of novel static optimizations that benefit from the semantics of high-level MLIR dialects to exploit the sparsity and structured control flow of generated code. Using these, LAGrad is able to achieve speedups of up to 2.8x and use 35x less memory relative to state of the art AD systems on real-world machine learning and computer vision benchmarks.
Swift and reliable critical load restoration (CLR) can help make a distribution system resilient towards extreme events. To optimally achieve that, alongside practical concerns such as limiting online computational bu...
详细信息
ISBN:
(纸本)9781665464413
Swift and reliable critical load restoration (CLR) can help make a distribution system resilient towards extreme events. To optimally achieve that, alongside practical concerns such as limiting online computational burden, some studies leverage model-free reinforcement learning (RL) to train control policies. Despite the advantages provided by RL algorithms, these approaches suffer from two issues: 1) the lack of a proper mechanism for constraint enforcement, and 2) poor sample efficiency. Therefore, in this paper, a primal-dual differentiable programming (PDDP) method is developed for guiding the training leading to a constraint-satisfying policy. Additionally, the model-based nature of the proposed method aims at improving sample efficiency. The experiment on a CLR problem demonstrates that PDDP can effectively train a control policy that both achieves desirable performance and satisfies required constraints.
In this work, we present novel machine learning and differentiable programming enhanced calibration techniques used to improve the energy resolution of the Silicon Drift Detectors (SDDs) of the VIP-2 underground exper...
详细信息
In this work, we present novel machine learning and differentiable programming enhanced calibration techniques used to improve the energy resolution of the Silicon Drift Detectors (SDDs) of the VIP-2 underground experiment at the Gran Sasso National Laboratory. We achieve for the first time a full width at half maximum in VIP-2 below 180 eV at 8 keV, improving around 10 eV on the previous state-of-the-art. SDDs energy resolution is a key parameter in the VIP-2 experiment, which is dedicated to searches for physics beyond the standard quantum theory, targeting Pauli exclusion principle violating atomic transitions. Additionally, we show that this method can correct for potential miscalibrations, requiring less fine-tuning with respect to standard methods.
Reduced order models with a-priori unknown closure relations are ubiquitous in transport problems. In this work, we present a machine-learning approach to finding closure relations utilising differentiable programming...
详细信息
Reduced order models with a-priori unknown closure relations are ubiquitous in transport problems. In this work, we present a machine-learning approach to finding closure relations utilising differentiable programming. We use the Su Olson radiation transport test problem as an example training data set. We present novel closures for second angular moment (variable Eddington factor), third angular moment and flux-limited diffusion models. We evaluate the improvement of the machine-learnt closures over those from the literature. These improvements are then tested by considering a modification to the Su Olson problem. Comparisons to literature closures show the machine learning models out-perform them in both the trained and unseen problems.
暂无评论