检索结果-内蒙古大学图书馆

A causal intervention framework for synthesizing mobility data and evaluating predictive neural networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hong, Ye Xin, Yanan Dirmeier, Simon Perez-Cruz, Fernando Raubal, Martin Institute of Cartography and Geoinformation ETH Zurich Switzerland Swiss Data Science Center ETH Zurich Switzerland EPFL Switzerland Institute for Machine Learning Department of Computer Science ETH Zurich Switzerland

Deep neural networks are increasingly utilized in mobility prediction tasks, yet their intricate internal workings pose challenges for interpretability, especially in comprehending how various aspects of mobility behavior affect predictions. This study introduces a causal intervention framework to assess the impact of mobility-related factors on neural networks designed for next location prediction – a task focusing on predicting the immediate next location of an individual. To achieve this, we employ individual mobility models to synthesize location visit sequences and control behavior dynamics by intervening in their data generation process. We evaluate the interventional location sequences using mobility metrics and input them into well-trained networks to analyze performance variations. The results demonstrate the effectiveness in producing location sequences with distinct mobility behaviors, thereby facilitating the simulation of diverse yet realistic spatial and temporal changes. These changes result in performance fluctuations in next location prediction networks, revealing impacts of critical mobility behavior factors, including sequential patterns in location transitions, proclivity for exploring new locations, and preferences in location choices at population and individual levels. The gained insights hold value for the real-world application of mobility prediction networks, and the framework is expected to promote the use of causal inference to enhance the interpretability and robustness of neural networks in mobility applications. © 2023, CC BY-NC-SA.

关键词： Prediction models

Computing the Bounds of the Number of Reticulations in a Tree-Child Network That Displays a Set of Trees

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Yufeng Zhang, Louxin Department of Computer Science and Engineering University of Connecticut StorrsCT06268 United States Department of Mathematics Center for Data Science and Machine Learning National University of Singapore Singapore119076 Singapore

Phylogenetic network is an evolutionary model that uses a rooted directed acyclic graph (instead of a tree) to model an evolutionary history of species in which reticulate events (e.g., hybrid speciation or horizontal gene transfer) occurred. Tree-child network is a kind of phylogenetic network with structural constraints. Existing approaches for tree-child network reconstruction can be slow for large data. In this paper, we present several computational approaches for bounding from below the number of reticulations in a tree-child network that displays a given set of rooted binary phylogenetic trees. In addition, we also present some theoretical results on bounding from above the number of reticulations. Through simulation, we demonstrate that the new lower bounds on the reticulation number for tree-child networks can practically be computed for large tree data. The bounds can provide estimates of reticulation for relatively large *** Codes 05C30 © 2023, CC BY.

关键词： Phylogenetic

Improving generalization and convergence by enhancing implicit regularization 24

学校读者我要写书评

暂无评论

Improving generalization and convergence by enhancing implic...

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Mingze Wang Jinbo Wang Haotian He Zilin Wang Guanhua Huang Feiyu Xiong Zhiyu Li Weinan E Lei Wu School of Mathematical Sciences Peking University and Institute for Advanced Algorithms Research (Shanghai) School of Mathematical Sciences Peking University School of Data Science University of Science and Technology of China and ByteDance Research Institute for Advanced Algorithms Research (Shanghai) School of Mathematical Sciences Peking University and Center for Machine Learning Research Peking University and Institute for Advanced Algorithms Research (Shanghai) and AI for Science Institute School of Mathematical Sciences Peking University and Center for Machine Learning Research Peking University

ISBN: (纸本)9798331314385

In this work, we propose an Implicit Regularization Enhancement (IRE) framework to accelerate the discovery of flat solutions in deep learning, thereby improving generalization and convergence. Specifically, IRE decouples the dynamics of flat and sharp directions, which boosts the sharpness reduction along flat directions while maintaining the training stability in sharp directions. We show that IRE can be practically incorporated with generic base optimizers without introducing significant computational overload. Experiments show that IRE consistently improves the generalization performance for image classification tasks across a variety of benchmark datasets (CIFAR-10/100, ImageNet) and models (ResNets and ViTs). Surprisingly, IRE also achieves a 2× speed-up compared to AdamW in the pre-training of Llama models (of sizes ranging from 60M to 229M) on datasets including Wikitext-103, Minipile, and Openwebtext. Moreover, we provide theoretical guarantees, showing that IRE can substantially accelerate the convergence towards flat minima in sharpness-aware minimization (SAM).

关键词：

Noisy Recovery in Unlimited Sampling via Adaptive Modulo Representations

学校读者我要写书评

暂无评论

Noisy Recovery in Unlimited Sampling via Adaptive Modulo Rep...

International Workshop on Compressed Sensing Theory and its Applications to Radar, Sonar and Remote Sensing (CoSeRa)

作者： Felipe Pagginelli Patricio Paul Catala Felix Krahmer Dept. of Mathematics TU Munich Garching Germany IBMI Helmholtz Munich Neuherberg Germany Dept. of Mathematics&Munich Data Science Institute TU Munich and Munich Center for Machine Learning Garching

ISBN: (数字)9798350365504

ISBN: (纸本)9798350365511

Recent works put forth the Unlimited Sensing Framework (USF), a novel approach to analog-to-digital conversion for high dynamic range sensing. It addresses the saturation phenomenon that commonly arises when physical measurements exceed the dynamic range of a sensor, yielding permanent loss of the input data. However, the USF still has some limitations when dealing with random noise. In the present paper, we propose a novel iterative method to tackle unlimited sensing in a noisy setting. In one step, our approach applies local transformations of the range to remove strong artifacts caused by the noise on local subdivisions of the domain. In the following step, the signal is then approximated via a least squares method. These two types of steps are then alternated. We illustrate the performances of our algorithm in high noise regime.

关键词： Multimodal sensors Conferences Noise Imaging Radar imaging Loss measurement High dynamic range Noise measurement Iterative methods Least mean squares methods

Multi-Scale Clinical-Guided Binocular Fusion Framework for Predicting New-Onset Hypertension Over a Four-Year Period

学校读者我要写书评

暂无评论

Multi-Scale Clinical-Guided Binocular Fusion Framework for P...

IEEE International Symposium on Biomedical Imaging

作者： Haoshen Li Zifan Chen Jie Zhao Heyun Chen Hexin Dong Mingze Yuan Bin Dong Li Zhang Center for Data Science Peking University China National Engineering Laboratory for Big Data Analysis and Applications Peking University China Peking University Changsha Institute for Computing and Digital Economy China Beijing International Center for Mathematical Research Peking University China Center for Machine Learning Research Peking University China

ISBN: (数字)9798350313338

ISBN: (纸本)9798350313345

Hypertension is a major global health concern, linked to various cardiovascular diseases and associated with distinct ocular manifestations. While recent advances in artificial intelligence have enabled accurate diagnosis of current hypertension through fundus images, predicting the future onset of hypertension remains an uncharted domain. In this study, we introduce the multi-scale clinical-guided binocular fusion framework (MCBO), designed to predict the likelihood of developing hypertension within the next four years. MCBO uniquely integrates left and right fundus images and clinical data, utilizing a shared-weight multi-stage Transformer-based encoder. Our multi-scale clinical-guided module (MCM) ensures image feature extraction is clinically contextualized based on clinical information, and our binocular fusion module (BFM) fuses binocular information. Comparative performance against seven baseline models establishes MCBO’s supremacy, with improvements of 6.7% in Area Under Curve (AUC), 6.9% in Accuracy (ACC), 5.1% in Sensitivity (SEN) and 5.5% in Specificity (SPE). This approach offers a promising avenue for proactive hypertension management, underscoring the potential of integrating Deep learning with clinical data for enhanced healthcare outcomes. Our code is available at https://***/HaoshenLi/MCBO.

关键词： Hypertension Deep learning Accuracy Sensitivity Codes Fuses Feature extraction

Audio-based Kinship Verification Using Age Domain Conversion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Qiyang Akman, Alican Jing, Xin Milling, Manuel Schuller, Björn W. GLAM Department of Computing Imperial College London United Kingdom MRI Technical University of Munich Germany MDSI – Munich Data Science Institute Munich Germany MCML – Munich Center for Machine Learning Munich Germany

Audio-based kinship verification (AKV) is important in many domains, such as home security monitoring, forensic identification, and social network analysis. A key challenge in the task arises from differences in age across samples from different individuals, which can be interpreted as a domain bias in a cross-domain verification task. To address this issue, we design the notion of an "age-standardised domain" wherein we utilise the optimised CycleGAN-VC3 network to perform age-audio conversion to generate the in-domain audio. The generated audio dataset is employed to extract a range of features, which are then fed into a metric learning architecture to verify kinship. Experiments are conducted on the KAN_AV audio dataset, which contains age and kinship labels. The results demonstrate that the method markedly enhances the accuracy of kinship verification, while also offering novel insights for future kinship verification *** Codes 68T10 © 2024, CC BY-NC-SA.

关键词： Economic and social effects

Enrolment-based personalisation for improving individual-level fairness in speech emotion recognition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Triantafyllopoulos, Andreas Schuller, Björn MRI Technical University of Munich Germany MCML - Munich Center for Machine Learning Germany MDSI - Munich Data Science Institute Germany GLAM - Group on Language Audio & Music Imperial College London United Kingdom

The expression of emotion is highly individualistic. However, contemporary speech emotion recognition (SER) systems typically rely on population-level models that adopt a 'one-size-fits-all' approach for predicting emotion. Moreover, standard evaluation practices measure performance also on the population level, thus failing to characterise how models work across different speakers. In the present contribution, we present a new method for capitalising on individual differences to adapt an SER model to each new speaker using a minimal set of enrolment utterances. In addition, we present novel evaluation schemes for measuring fairness across different speakers. Our findings show that aggregated evaluation metrics may obfuscate fairness issues on the individual-level, which are uncovered by our evaluation, and that our proposed method can improve performance both in aggregated and disaggregated terms. © 2024, CC BY-NC-ND.

关键词： Emotion Recognition

A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Marks, Markus Knott, Manuel Kondapaneni, Neehar Cole, Elijah Defraeye, Thijs Perez-Cruz, Fernando Perona, Pietro California Institute of Technology United States ETH Zurich Institute for Machine Learning Department of Computer Science Switzerland Swiss Data Science Center ETH Zurich and EPFL Switzerland Empa Swiss Federal Laboratories for Materials Science and Technology Switzerland Altos Labs Switzerland

Self-supervised learning (SSL) is a machine learning approach where the data itself provides supervision, eliminating the need for external labels. The model is forced to learn about the data's inherent structure or context by solving a pretext task. With SSL, models can learn from abundant and cheap unlabeled data, significantly reducing the cost of training models where labels are expensive or inaccessible. In Computer Vision, SSL is widely used as pre-training followed by a downstream task, such as supervised transfer, few-shot learning on smaller labeled data sets, and/or unsupervised clustering. Unfortunately, it is infeasible to evaluate SSL methods on all possible downstream tasks and objectively measure the quality of the learned representation. Instead, SSL methods are evaluated using in-domain evaluation protocols, such as fine-tuning, linear probing, and k-nearest neighbors (kNN). However, it is not well understood how well these evaluation protocols estimate the representation quality of a pre-trained model for different downstream tasks under different conditions, such as dataset, metric, and model architecture. In this work, we study how classification-based evaluation protocols for SSL correlate and how well they predict downstream performance on different dataset types. Our study includes eleven common image datasets and 26 models that were pre-trained with different SSL methods or have different model backbones. We find that in-domain linear/kNN probing protocols are, on average, the best general predictors for out-of-domain performance. We further investigate the importance of batch normalization for the various protocols and evaluate how robust correlations are for different kinds of dataset domain shifts. In addition, we challenge assumptions about the relationship between discriminative and generative self-supervised methods, finding that most of their performance differences can be explained by changes to model backbones. Copyright © 2024, The

关键词： Nearest neighbor search

The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bonfanti, Andrea Bruno, Giuseppe Cipriani, Cristina BMW AG Basque Center for Applied Mathematics University of the Basque Country Digital Campus Munich Spain BMW AG Digital Campus Munich Germany Technical University of Munich Munich Center for Machine Learning Munich Data Science Institute Germany

The Neural Tangent Kernel (NTK) viewpoint is widely employed to analyze the training dynamics of overparameterized Physics-Informed Neural Networks (PINNs). However, unlike the case of linear Partial Differential Equations (PDEs), we show how the NTK perspective falls short in the nonlinear scenario. Specifically, we establish that the NTK yields a random matrix at initialization that is not constant during training, contrary to conventional belief. Another significant difference from the linear regime is that, even in the idealistic infinite-width limit, the Hessian does not vanish and hence it cannot be disregarded during training. This motivates the adoption of second-order optimization methods. We explore the convergence guarantees of such methods in both linear and nonlinear cases, addressing challenges such as spectral bias and slow convergence. Every theoretical result is supported by numerical examples with both linear and nonlinear PDEs, and we highlight the benefits of second-order methods in benchmark test cases. © 2024, CC BY.

关键词： Numerical methods