检索结果-内蒙古大学图书馆

Gradient is All You Need?

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Riedl, Konstantin Klock, Timo Geldhauser, Carina Fornasier, Massimo Technical University of Munich Munich Germany Munich Center for Machine Learning Munich Germany Deeptech Consulting Oslo Norway Munich Data Science Institute Munich Germany

In this paper we provide a novel analytical perspective on the theoretical understanding of gradient-based learning algorithms by interpreting consensus-based optimization (CBO), a recently proposed multi-particle derivative-free optimization method, as a stochastic relaxation of gradient descent. Remarkably, we observe that through communication of the particles, CBO exhibits a stochastic gradient descent (SGD)-like behavior despite solely relying on evaluations of the objective function. The fundamental value of such link between CBO and SGD lies in the fact that CBO is provably globally convergent to global minimizers for ample classes of nonsmooth and nonconvex objective functions, hence, on the one side, offering a novel explanation for the success of stochastic relaxations of gradient descent. On the other side, contrary to the conventional wisdom for which zero-order methods ought to be inefficient or not to possess generalization abilities, our results unveil an intrinsic gradient descent nature of such heuristics. This viewpoint furthermore complements previous insights into the working principles of CBO, which describe the dynamics in the mean-field limit through a nonlinear nonlocal partial differential equation that allows to alleviate complexities of the nonconvex function landscape. Our proofs leverage a completely nonsmooth analysis, which combines a novel quantitative version of the Laplace principle (log-sum-exp trick) and the minimizing movement scheme (proximal iteration). In doing so, we furnish useful and precise insights that explain how stochastic perturbations of gradient descent overcome energy barriers and reach deep levels of nonconvex functions. Instructive numerical illustrations support the provided theoretical insights. Copyright © 2023, The Authors. All rights reserved.

关键词： Optimization

Decoupling Common and Unique Representations for Multimodal Self-supervised learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Yi Albrecht, Conrad M. Braham, Nassim Ait Ali Liu, Chenying Xiong, Zhitong Zhu, Xiao Xiang Data Science in Earth Observation Technical University of Munich Germany Remote Sensing Technology Institute German Aerospace Center Germany Munich Center for Machine Learning Germany

The increasing availability of multi-sensor data sparks wide interest in multimodal self-supervised learning. However, most existing approaches learn only common representations across modalities while ignoring intra-modal training and modality-unique representations. We propose Decoupling Common and Unique Representations (DeCUR), a simple yet effective method for multimodal self-supervised learning. By distinguishing inter- and intra-modal embeddings through multimodal redundancy reduction, DeCUR can integrate complementary information across different modalities. We evaluate DeCUR in three common multimodal scenarios (radar-optical, RGB-elevation, and RGB-depth), and demonstrate its consistent improvement regardless of architectures and for both multimodal and modality-missing settings. With thorough experiments and comprehensive analysis, we hope this work can provide valuable insights and raise more interest in researching the hidden relationships of multimodal representations4 Copyright © 2023, The Authors. All rights reserved.

关键词： Self-supervised learning

Personalised Speech-Based PTSD Prediction Using Weighted-Instance learning 46

学校读者我要写书评

暂无评论

Personalised Speech-Based PTSD Prediction Using Weighted-Ins...

46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2024

作者： Kathan, Alexander Amiriparian, Shahin Triantafyllopoulos, Andreas Gebhard, Alexander Milkus, Sabrina Hohmann, Jonas Muderlak, Pauline Schottdorf, Jürgen Musil, Richard Schuller, Björn W. University of Augsburg Eihw - Embedded Intelligence for Healthcare and Wellbeing Germany Mri Technical Univsersity of Munich Chi - Health Informatics Germany Mcml - Munich Center for Machine Learning Germany University Hospital Lmu Munich Department of Psychiatry and Psychotherapy Germany Zentrumspraxis Friedberg Germany Imperial College London Glam - Group on Language Audio & Music United Kingdom Mdsi - Munich Data Science Institute Germany

ISBN: (纸本)9798350371499

Post-traumatic stress disorder (PTSD) is a prevalent disorder that can develop in people who have experienced very stressful, shocking, or distressing events. It has great influence on peoples' daily life and can affect their mental, physical, or social wellbeing, which is why a timely and professional treatment is required. In this paper, we propose a personalised speech-based PTSD prediction approach using a newly collected dataset which consists of 15 participants, including speech recordings from people with PTSD and healthy controls. In addition, the dataset includes data before and after a clinical intervention so that the prediction can be analysed at different points in time. In our experiments, we demonstrate the superiority of the personalised approach, achieving a best area under the ROC curve (AUC) of 82 % and a best relative improvement of 7 % points compared to the non-personalised model. © 2024 IEEE.

关键词： Prediction models

TOWARDS DIVERSE EVALUATION OF CLASS INCREMENTAL learning: A REPRESENTATION learning PERSPECTIVE 3

学校读者我要写书评

暂无评论

TOWARDS DIVERSE EVALUATION OF CLASS INCREMENTAL LEARNING: A ...

3rd Conference on Lifelong learning Agents, CoLLAs 2024

作者： Cha, Sungmin Kwak, Jihwan Shim, Dongsub Kim, Hyunwoo Lee, Moontae Lee, Honglak Moon, Taesup Computer Science Department Courant Institute of Mathematical Sciences New York University United States Department of Electrical and Computer Engineering Seoul National University Korea Republic of Advanced Machine Learning Lab LG AI Research Korea Republic of Zhejiang Lab China Department of Information and Decision Sciences University of Illinois Chicago United States ASRI INMC IPAI AIIS Seoul National University Korea Republic of

Class incremental learning (CIL) algorithms aim to continually learn new object classes from incrementally arriving data while not forgetting past learned classes. The common evaluation protocol for CIL algorithms is to measure the average test accuracy across all classes learned so far - however, we argue that solely focusing on maximizing the test accuracy may not necessarily lead to developing a CIL algorithm that also continually learns and updates the representations, which may be transferred to the downstream tasks. To that end, we experimentally analyze neural network models trained by CIL algorithms using various evaluation protocols in representation learning and propose new analysis methods. Our experiments show that most state-of-the-art algorithms prioritize high stability and do not significantly change the learned representation, and sometimes even learn a representation of lower quality than a naive baseline. However, we observe that these algorithms can still achieve high test accuracy because they enable a model to learn a classifier that closely resembles an estimated linear classifier trained for linear probing. Furthermore, the base model learned in the first task, which involves single-task learning, exhibits varying levels of representation quality across different algorithms, and this variance impacts the final performance of CIL algorithms. Therefore, we suggest that the representation-level evaluation should be considered as an additional recipe for more diverse evaluation for CIL algorithms. © 2024 CoLLAs. All Rights Reserved.

关键词： Contrastive learning

CBAM-SAUNet: A novel attention U-Net for effective segmentation of corner cases 46

学校读者我要写书评

暂无评论

CBAM-SAUNet: A novel attention U-Net for effective segmentat...

46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2024

作者： Rajamani, Srividya Tirunellai Rajamani, Kumar Angeline, J. Karthika, R. Schuller, Björn W. Embedded Intelligence for Health Care & Wellbeing University of Augsburg Germany Department of Artificial Intelligence Marwadi University Gujarat Rajkot India Department of Electronics and Communication Engineering Amrita School of Engineering Amrita Vishwa Vidyapeetham Coimbatore India Germany Munich Data Science Institute Germany Munich Center for Machine Learning GLAM-the Group on Language Audio & Music Imperial College London London United Kingdom

ISBN: (纸本)9798350371499

U-Net has been demonstrated to be effective for the task of medical image segmentation. Additionally, integrating attention mechanism into U-Net has been shown to yield significant benefits. The Shape Attentive U-Net (SAUNet) is one such recently proposed attention U-Net that also focuses on interpretability. Furthermore, recent research has focused on identification and reporting of corner cases in segmentation to accelerate the utilisation of deep learning models in clinical practise. However, achieving good model performance on such corner cases is a less-explored research area. In this paper, we propose CBAM-SAUNet which enhances the dual attention decoder block of SAUNet to improve its performance on corner cases. We achieve this by utilising a novel variant of the Convolutional Block Attention Module (CBAM)'s channel attention in the decoder block of SAUNet. We demonstrate the effectiveness of CBAM-SAUNet in the Automated Cardiac Diagnosis Challenge (ACDC) cardiac MRI segmentation challenge. Our proposed novel approach results in improvement in the Dice scores of 12% for Left Ventricle (LV) as well as Right Ventricle (RV) segmentation and 8% for Myocardium (MYO) for the identified corner-case dataset. © 2024 IEEE.

关键词： Image segmentation

Model and Feature Diversity for Bayesian Neural Networks in Mutual learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Pham, Cuong Nguyen, Cuong C. Le, Trung Phung, Dinh Carneiro, Gustavo Do, Thanh-Toan Department of Data Science and AI Monash University Australia Australian Institute for Machine Learning University of Adelaide Australia Centre for Vision Speech and Signal Processing University of Surrey United Kingdom VinAI Viet Nam

Bayesian Neural Networks (BNNs) offer probability distributions for model parameters, enabling uncertainty quantification in predictions. However, they often underperform compared to deterministic neural networks. Utilizing mutual learning can effectively enhance the performance of peer BNNs. In this paper, we propose a novel approach to improve BNNs performance through deep mutual learning. The proposed approaches aim to increase diversity in both network parameter distributions and feature distributions, promoting peer networks to acquire distinct features that capture different characteristics of the input, which enhances the effectiveness of mutual learning. Experimental results demonstrate significant improvements in the classification accuracy, negative log-likelihood, and expected calibration error when compared to traditional mutual learning for BNNs. © 2024, CC BY.

关键词： Probability distributions

Less is More: Facial Landmarks can Recognize a Spontaneous Smile 33

学校读者我要写书评

暂无评论

Less is More: Facial Landmarks can Recognize a Spontaneous S...

33rd British machine Vision Conference Proceedings, BMVC 2022

作者： Tushar, Md Tahrim Faroque Yang, Yan Hossain, Md Zakir Naim, Sheikh Motahar Mohammed, Nabeel Rahman, Shafin Department of Electrical and Computer Engineering North South University Bangladesh Biological Data Science Institute The Australian National University Canberra Australia CSIRO Agriculture and Food Canberra Australia CSIRO Machine Learning and Artificial Intelligence Future Science Platform Canberra Australia Amazon Web Services

Smile veracity classification is a task of interpreting social interactions. Broadly, it distinguishes between spontaneous and posed smiles. Previous approaches used hand-engineered features from facial landmarks or considered raw smile videos in an end-to-end manner to perform smile classification tasks. Feature-based methods require intervention from human experts on feature engineering and heavy pre-processing steps. On the contrary, raw smile video inputs fed into end-to-end models bring more automation to the process with the cost of considering many redundant facial features (beyond landmark locations) that are mainly irrelevant to smile veracity classification. It remains unclear to establish discriminative features from landmarks in an end-to-end manner. We present a MeshSmileNet framework, a transformer architecture, to address the above limitations. To eliminate redundant facial features, our landmarks input is extracted from Attention Mesh, a pre-trained landmark detector. Again, to discover discriminative features, we consider the relativity and trajectory of the landmarks. For the relativity, we aggregate facial landmark that conceptually formats a curve at each frame to establish local spatial features. For the trajectory, we estimate the movements of landmark composed features across time by self-attention mechanism, which captures pairwise dependency on the trajectory of the same landmark. This idea allows us to achieve state-of-the-art performances on UVA-NEMO, BBC, MMI Facial Expression, and SPOS datasets. © 2022. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.

关键词： Relativity

Uni-ELF: A Multi-Level Representation learning Framework for Electrolyte Formulation Design

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zeng, Boshen Chen, Sian Liu, Xinxin Chen, Changhong Deng, Bin Wang, Xiaoxu Gao, Zhifeng Zhang, Yuzhi Weinan, E. Zhang, Linfeng DP Technology Beijing100080 China Peking University Beijing100871 China AI for Science Institute Beijing100080 China Center for Machine Learning Research School of Mathematical Sciences Peking University Beijing China

Advancements in lithium battery technology heavily rely on the design and engineering of electrolytes. However, current schemes for molecular design and recipe optimization of electrolytes lack an effective computational-experimental closed loop and often fall short in accurately predicting diverse electrolyte formulation properties. In this work, we introduce Uni-ELF, a novel multi-level representation learning framework to advance electrolyte design. Our approach involves two-stage pretraining: reconstructing three-dimensional molecular structures at the molecular level using the Uni-Mol model, and predicting statistical structural properties (e.g., radial distribution functions) from molecular dynamics simulations at the mixture level. Through this comprehensive pretraining, Uni-ELF is able to capture intricate molecular and mixture-level information, which significantly enhances its predictive capability. As a result, Uni-ELF substantially outperforms state-of-the-art methods in predicting both molecular properties (e.g., melting point, boiling point, synthesizability) and formulation properties (e.g., conductivity, Coulombic efficiency). Moreover, Uni-ELF can be seamlessly integrated into an automatic experimental design workflow. We believe this innovative framework will pave the way for automated AI-based electrolyte design and engineering. Copyright © 2024, The Authors. All rights reserved.

关键词： Electrolytes

U-Star: AN Asymmetric U-Shaped Network Based on Element-Wise Multiplication to Segment Nuclei in H&E Stained Histological Images

学校读者我要写书评

暂无评论

U-Star: AN Asymmetric U-Shaped Network Based on Element-Wise...

IEEE International Symposium on Biomedical Imaging

作者： Guangzhengao Yang Li Zhang Jie Zhao Zifan Chen Haoshen Li Bin Dong Center for Data Science Peking University China National Engineering Laboratory for Big Data Analysis and Applications Peking University China Peking University Changsha Institute for Computing and Digital Economy China Beijing International Center for Mathematical Research Peking University China Center for Machine Learning Research Peking University China National Biomedical Imaging Center Peking University China

ISBN: (数字)9798331520526

ISBN: (纸本)9798331520533

Nuclei segmentation in Hematoxylin and Eosin (H&E) stained images plays a crucial role in cancer diagnosis and pathological evaluation, enabling pathologists to identify abnormal cells and assess their morphology and distribution. While current automated nuclei segmentation methods predominantly employ convolutional neural networks and attention mechanisms, the potential of element-wise multiplication has been largely unexplored. This paper introduces U-Star, a novel asymmetric segmentation network based on the star block that leverages element-wise multiplication. U-Star adopts the classic encoder-decoder architecture and innovatively implements star-connection as an alternative to traditional skip-connections. In experiments on an H&E stained image dataset, U-Star achieved superior performance with a Dice coefficient of 0.8783, accuracy of 0.9089, and IoU of 0.7929, significantly outperforming baseline models. Extensive ablation studies validate the effectiveness of the star-connection and demonstrate the advantages of our proposed framework. Beyond advancing the application of element-wise multiplication techniques, U-Star shows promising potential for broader applications in medical image segmentation.

关键词： Image segmentation Pathology Technological innovation Image analysis Stars Morphology Network architecture Medical diagnostic imaging Tuning Cancer