检索结果-内蒙古大学图书馆

Sufficient dimension reduction for high-dimensional regression and low-dimensional embedding: tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ghojogh, Benyamin Ghodsi, Ali Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Department of Statistics and Actuarial Science & David R. Cheriton School of Computer Science Data Analytics Laboratory University of Waterloo WaterlooON Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This is a tutorial and survey paper on various methods for Sufficient Dimension Reduction (SDR). We cover these methods with both statistical high-dimensional regression perspective and machine learning approach for dimensionality reduction. We start with introducing inverse regression methods including Sliced Inverse Regression (SIR), Sliced Average Variance Estimation (SAVE), contour regression, directional regression, Principal Fitted Components (PFC), Likelihood Acquired Direction (LAD), and graphical regression. Then, we introduce forward regression methods including Principal Hessian Directions (pHd), Minimum Average Variance Estimation (MAVE), Conditional Variance Estimation (CVE), and deep SDR methods. Finally, we explain Kernel Dimension Reduction (KDR) both for supervised and unsupervised learning. We also show that supervised KDR and supervised PCA are equivalent. Copyright © 2021, The Authors. All rights reserved.

关键词： Regression analysis

Generative Adversarial Networks and adversarial autoencoders: Tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

This is a tutorial and survey paper on Generative Adversarial Network (GAN), adversarial autoencoders, and their variants. We start with explaining adversarial learning and the vanilla GAN. Then, we explain the conditional GAN and DCGAN. The mode collapse problem is introduced and various methods, including minibatch GAN, unrolled GAN, BourGAN, mixture GAN, D2GAN, and Wasserstein GAN, are introduced for resolving this problem. Then, maximum likelihood estimation in GAN are explained along with f-GAN, adversarial variational Bayes, and Bayesian GAN. Then, we cover feature matching in GAN, InfoGAN, GRAN, LSGAN, energy-based GAN, CatGAN, MMD GAN, LapGAN, progressive GAN, triple GAN, LAG, GMAN, AdaGAN, CoGAN, inverse GAN, BiGAN, ALI, SAGAN, Few-shot GAN, SinGAN, and interpolation and evaluation of GAN. Then, we introduce some applications of GAN such as image-to-image translation (including PatchGAN, CycleGAN, DeepFaceDrawing, simulated GAN, interactive GAN), text-to-image translation (including StackGAN), and mixing image characteristics (including FineGAN and MixNMatch). Finally, we explain the autoencoders based on adversarial learning including adversarial autoencoder, PixelGAN, and implicit autoencoder. Copyright © 2021, The Authors. All rights reserved.

关键词： Generative adversarial networks

KKT conditions, first-order and second-order optimization, and distributed optimization: Tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ghojogh, Benyamin Ghodsi, Ali Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Department of Statistics and Actuarial Science & David R. Cheriton School of Computer Science Data Analytics Laboratory University of Waterloo WaterlooON Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This is a tutorial and survey paper on Karush-Kuhn-Tucker (KKT) conditions, first-order and second-order numerical optimization, and distributed optimization. After a brief review of history of optimization, we start with some preliminaries on properties of sets, norms, functions, and concepts of optimization. Then, we introduce the optimization problem, standard optimization problems (including linear programming, quadratic programming, and semidefinite programming), and convex problems. We also introduce some techniques such as eliminating inequality, equality, and set constraints, adding slack variables, and epigraph form. We introduce Lagrangian function, dual variables, KKT conditions (including primal feasibility, dual feasibility, weak and strong duality, complementary slackness, and stationarity condition), and solving optimization by method of Lagrange multipliers. Then, we cover first-order optimization including gradient descent, line-search, convergence of gradient methods, momentum, steepest descent, and backpropagation. Other first-order methods are explained, such as accelerated gradient method, stochastic gradient descent, mini-batch gradient descent, stochastic average gradient, stochastic variance reduced gradient, AdaGrad, RMSProp, and Adam optimizer, proximal methods (including proximal mapping, proximal point algorithm, and proximal gradient method), and constrained gradient methods (including projected gradient method, projection onto convex sets, and Frank-Wolfe method). We also cover non-smooth and `1 optimization methods including lasso regularization, convex conjugate, Huber function, soft-thresholding, coordinate descent, and subgradient methods. Then, we explain second-order methods including Newton’s method for unconstrained, equality constrained, and inequality constrained problems. We explain the interior-point method, barrier methods, Wolfe conditions for line-search, fast solving system of equations (including decomposition methods a

关键词： Quadratic programming

Reproducing kernel Hilbert space, Mercer's theorem, eigenfunctions, Nyström method, and use of kernels in machine learning: Tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

This is a tutorial and survey paper on kernels, kernel methods, and related fields. We start with reviewing the history of kernels in functional analysis and machine learning. Then, Mercer kernel, Hilbert and Banach spaces, Reproducing Kernel Hilbert Space (RKHS), Mercer's theorem and its proof, frequently used kernels, kernel construction from distance metric, important classes of kernels (including bounded, integrally positive definite, universal, stationary, and characteristic kernels), kernel centering and normalization, and eigenfunctions are explained in detail. Then, we introduce types of use of kernels in machine learning including kernel methods (such as kernel support vector machines), kernel learning by semi-definite programming, Hilbert-Schmidt independence criterion, maximum mean discrepancy, kernel mean embedding, and kernel dimensionality reduction. We also cover rank and factorization of kernel matrix as well as the approximation of eigenfunctions and kernels using the Nyström method. This paper can be useful for various fields of science including machine learning, dimensionality reduction, functional analysis in mathematics, and mathematical physics in quantum mechanics. Copyright © 2021, The Authors. All rights reserved.

关键词： Functional analysis

Uniform manifold approximation and projection (UMAP) and its variants: Tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

Uniform Manifold Approximation and Projection (UMAP) is one of the state-of-the-art methods for dimensionality reduction and data visualization. This is a tutorial and survey paper on UMAP and its variants. We start with UMAP algorithm where we explain probabilities of neighborhood in the input and embedding spaces, optimization of cost function, training algorithm, derivation of gradients, and supervised and semisupervised embedding by UMAP. Then, we introduce the theory behind UMAP by algebraic topology and category theory. Then, we introduce UMAP as a neighbor embedding method and compare it with t-SNE and LargeVis algorithms. We discuss negative sampling and repulsive forces in UMAP's cost function. DensMAP is then explained for density-preserving embedding. We then introduce parametric UMAP for embedding by deep learning and progressive UMAP for streaming and out-of-sample data embedding. Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey

学校读者我要写书评

暂无评论

arXiv 2021年

This is a tutorial and survey paper for nonlinear dimensionality and feature extraction methods which are based on the Laplacian of graph of data. We first introduce adjacency matrix, definition of Laplacian matrix, and the interpretation of Laplacian. Then, we cover the cuts of graph and spectral clustering which applies clustering in a subspace of data. Different optimization variants of Laplacian eigenmap and its out-of-sample extension are explained. Thereafter, we introduce the locality preserving projection and its kernel variant as linear special cases of Laplacian eigenmap. Versions of graph embedding are then explained which are generalized versions of Laplacian eigenmap and locality preserving projection. Finally, diffusion map is introduced which is a method based on Laplacian of data and random walks on the data graph. Copyright © 2021, The Authors. All rights reserved.

关键词： Surveys

Johnson-lindenstrauss lemma, linear and nonlinear random projections, random fourier features, and random kitchen sinks: Tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ghojogh, Benyamin Ghodsi, Ali Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Department of Statistics and Actuarial Science & David R. Cheriton School of Computer Science Data Analytics Laboratory University of Waterloo WaterlooON Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This is a tutorial and survey paper on the Johnson-Lindenstrauss (JL) lemma and linear and nonlinear random projections. We start with linear random projection and then justify its correctness by JL lemma and its proof. Then, sparse random projections with `1 norm and interpolation norm are introduced. Two main applications of random projection, which are low-rank matrix approximation and approximate nearest neighbor search by random projection onto hypercube, are explained. Random Fourier Features (RFF) and Random Kitchen Sinks (RKS) are explained as methods for nonlinear random projection. Some other methods for nonlinear random projection, including extreme learning machine, randomly weighted neural networks, and ensemble of random projections, are also introduced. Copyright © 2021, The Authors. All rights reserved.

关键词： Surveys

Neuron-astrocyte coupling in lateral habenula mediates depressive-like behaviors

学校读者我要写书评

暂无评论

Cell 2025年第12期188卷 3291-3309.e24,27页

作者： Xin, Qianqian Wang, Junying Zheng, Jinkun Tan, Yi Jia, Xiaoning Ni, Zheyi Xu, Zijie Feng, Jiesi Wu, Zhaofa Li, Yulong Li, Xiao-Ming Ma, Huan Hu, Hailan Department of Neurobiology Affiliated Mental Health Center & Hangzhou Seventh People's Hospital Zhejiang University School of Medicine Hangzhou 310058 China Nanhu Brain-Computer Interface Institute Liangzhu Laboratory MOE Frontier Science Center for Brain Science and Brain-Machine Integration State Key Laboratory of Brain-Machine Intelligence New Cornerstone Science Laboratory Zhejiang University Hangzhou 311121 China Department of Psychiatry and International Institutes of Medicine The Fourth Affiliated Hospital Zhejiang University School of Medicine Yiwu 322000 China State Key Laboratory of Membrane Biology Peking University School of Life Sciences Beijing 100871 China

The lateral habenula (LHb) neurons and astrocytes have been strongly implicated in depression etiology, but it was not clear how the two dynamically interact during depression onset. Here, using multi-brain-region calcium photometry recording in freely moving mice, we discover that stress induces a most rapid astrocytic calcium rise and a bimodal neuronal response in the LHb. LHb astrocytic calcium requires the α_1A-adrenergic receptor and depends on a recurrent neural network between the LHb and locus coeruleus (LC). Through the gliotransmitter glutamate and ATP/adenosine, LHb astrocytes mediate the second-wave LHb neuronal activation and norepinephrine (NE) release. Activation or inhibition of LHb astrocytic calcium signaling facilitates or prevents stress-induced depressive-like behaviors, respectively. These results identify a stress-induced positive feedback loop in the LHb-LC axis, with astrocytes being a critical signaling relay. The identification of this prominent neuron-glia interaction may shed light on stress management and depression prevention. © 2025 Elsevier Inc.

关键词： astrocytes depression lateral habenula locus coeruleus norepinephrine phasic and tonic NE reboxetine SNRI stress α1A-adrenergic receptor

Touchstone benchmark: are we on the right way for evaluating AI algorithms for medical segmentation? 24

学校读者我要写书评

暂无评论

Touchstone benchmark: are we on the right way for evaluating...

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Pedro R. A. S. Bassi Wenxuan Li Yucheng Tang Fabian Isensee Zifu Wang Jieneng Chen Yu-Cheng Chou Saikat Roy Yannick Kirchhoff Maximilian Rokuss Ziyan Huang Jin Ye Junjun He Tassilo Wald Constantin Ulrich Michael Baumgartner Klaus H. Maier-Hein Paul Jaeger Yiwen Ye Yutong Xie Jianpeng Zhang Ziyang Chen Yong Xia Zhaohu Xing Lei Zhu Yousef Sadegheih Afshin Bozorgpour Pratibha Kumari Reza Azad Dorit Merhof Pengcheng Shi Ting Ma Yuxin Du Fan Bai Tiejun Huang Bo Zhao Haonan Wang Xiaomeng Li Hanxue Gu Haoyu Dong Jichen Yang Maciej A. Mazurowski Saumya Gupta Linshan Wu Jiaxin Zhuang Hao Chen Holger Roth Daguang Xu Matthew B. Blaschko Sergio Decherchi Andrea Cavalli Alan L. Yuille Zongwei Zhou Department of Computer Science Johns Hopkins University and Department of Pharmacy and Biotechnology University of Bologna and Center for Biomolecular Nanotechnologies Istituto Italiano di Tecnologia Department of Computer Science Johns Hopkins University NVIDIA Division of Medical Image Computing German Cancer Research Center (DKFZ) and Helmholtz Imaging German Cancer Research Center (DKFZ) ESAT-PSI KU Leuven Division of Medical Image Computing German Cancer Research Center (DKFZ) and Faculty of Mathematics and Computer Science Heidelberg University Division of Medical Image Computing German Cancer Research Center (DKFZ) and Faculty of Mathematics and Computer Science Heidelberg University and HIDSS4Health - Helmholtz Information and Data Science School for Health Shanghai Jiao Tong University Shanghai Artificial Intelligence Laboratory Division of Medical Image Computing German Cancer Research Center (DKFZ) Division of Medical Image Computing German Cancer Research Center (DKFZ) and Pattern Analysis and Learning Group Department of Radiation Oncology Heidelberg University Hospital Helmholtz Imaging German Cancer Research Center (DKFZ) and Interactive Machine Learning Group (IML) DKFZ School of Computer Science and Engineering Northwestern Polytechnical University Australian Institute for Machine Learning The University of Adelaide College of Computer Science and Technology Zhejiang University Hong Kong University of Science and Technology (Guangzhou) Hong Kong University of Science and Technology (Guangzhou) and Hong Kong University of Science and Technology Faculty of Informatics and Data Science University of Regensburg Faculty of Electrical Engineering and Information Technology RWTH Aachen University Faculty of Informatics and Data Science University of Regensburg and Fraunhofer Institute for Digital Medicine MEVIS Electronic & Information Engineering School Harbin Institute of Technology (Shenzhen) Shanghai Jiao Tong University and Beijing Academy of Artificial Intelligence (BAAI) S

ISBN: (纸本)9798331314385

How can we test AI performance? This question seems trivial, but it isn't. Standard benchmarks often have problems such as in-distribution and small-size test sets, oversimplified metrics, unfair comparisons, and short-term outcome pressure. As a consequence, good performance on standard benchmarks does not guarantee success in real-world scenarios. To address these problems, we present Touchstone, a large-scale collaborative segmentation benchmark of 9 types of abdominal organs. This benchmark is based on 5,195 training CT scans from 76 hospitals around the world and 5,903 testing CT scans from 11 additional hospitals. This diverse test set enhances the statistical significance of benchmark results and rigorously evaluates AI algorithms across out-of-distribution scenarios. We invited 14 inventors of 19 AI algorithms to train their algorithms, while our team, as a third party, independently evaluated these algorithms. In addition, we also evaluated pre-existing AI frameworks—which, differing from algorithms, are more flexible and can support different algorithms—including MONAI from NVIDIA, nnU-Net from DKFZ, and numerous other open-source frameworks. We are committed to expanding this benchmark to encourage more innovation of AI algorithms for the medical domain.

关键词：