检索结果-内蒙古大学图书馆

Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI 41

学校读者我要写书评

暂无评论

Position: Bayesian Deep Learning is Needed in the Age of Lar...

41st International Conference on Machine Learning, ICML 2024

作者： Papamarkou, Theodore Skoularidou, Maria Palla, Konstantina Aitchison, Laurence Arbel, Julyan Dunson, David Filippone, Maurizio Fortuin, Vincent Hennig, Philipp Hernández-Lobato, José Miguel Hubin, Aliaksandr Immer, Alexander Karaletsos, Theofanis Khan, Mohammad Emtiyaz Kristiadi, Agustinus Li, Yingzhen Mandt, Stephan Nemeth, Christopher Osborne, Michael A. Rudner, Tim G.J. Rügamer, David Teh, Yee Whye Welling, Max Wilson, Andrew Gordon Zhang, Ruqi Department of Mathematics The University of Manchester Manchester United Kingdom Eric and Wendy Schmidt Center Broad Institute of MIT and Harvard Cambridge United States Spotify London United Kingdom Computational Neuroscience Unit University of Bristol Bristol United Kingdom Centre Inria de l'Université Grenoble Alpes Grenoble France Department of Statistical Science Duke University United States Statistics Program KAUST Saudi Arabia Helmholtz AI Munich Germany Department of Computer Science Technical University of Munich Munich Germany Munich Center for Machine Learning Munich Germany Tübingen AI Center University of Tübingen Tübingen Germany Department of Engineering University of Cambridge Cambridge United Kingdom Department of Mathematics University of Oslo Oslo Norway Bioinformatics and Applied Statistics Norwegian University of Life Sciences Ås Norway Department of Computer Science ETH Zurich Switzerland Chan Zuckerberg Initiative CA United States Center for Advanced Intelligence Project RIKEN Tokyo Japan Vector Institute Toronto Canada Department of Computing Imperial College London London United Kingdom Department of Computer Science UC Irvine Irvine United States Department of Mathematics and Statistics Lancaster University Lancaster United Kingdom Department of Engineering Science University of Oxford Oxford United Kingdom Center for Data Science New York University New York United States Department of Statistics LMU Munich Munich Germany DeepMind London United Kingdom Department of Statistics University of Oxford Oxford United Kingdom Informatics Institute University of Amsterdam Amsterdam Netherlands Courant Institute of Mathematical Sciences Center for Data Science Computer Science Department New York University New York United States Department of Computer Science Purdue University West Lafayette United States

In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets. However, a broader perspective reveals a multitude of overlooked metrics, tasks, and data types, such as uncertainty, active and continual learning, and scientific data, that demand attention. Bayesian deep learning (BDL) constitutes a promising avenue, offering advantages across these diverse settings. This paper posits that BDL can elevate the capabilities of deep learning. It revisits the strengths of BDL, acknowledges existing challenges, and highlights some exciting research avenues aimed at addressing these obstacles. Looking ahead, the discussion focuses on possible ways to combine large-scale foundation models with BDL to unlock their full potential. Copyright 2024 by the author(s)

关键词： Contrastive Learning

Reproducing kernel Hilbert C*-module and kernel mean embeddings

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2021年第1期22卷 12292-12347页

作者： Yuka Hashimoto Isao Ishikawa Masahiro Ikeda Fuyuta Komura Takeshi Katsura Yoshinobu Kawahara NTT Network Service Systems Laboratories NTT Corporation Midori-cho Musashinoshi Tokyo Japan and Graduate School of Science and Technology Keio University Hiyoshi Kohoku Yokohama Kanagawa Japan Center for Data Science Ehime University Bunkyo-cho Matsuyama Ehime Japan and Center for Advanced Intelligence Project RIKEN Nihonbashi Chuo-ku Tokyo Japan Center for Advanced Intelligence Project RIKEN Nihonbashi Chuo-ku Tokyo Japan and Faculty of Science and Technology Keio University Hiyoshi Kohoku Yokohama Kanagawa Japan Faculty of Science and Technology Keio University Hiyoshi Kohoku Yokohama Kanagawa Japan and Center for Advanced Intelligence Project RIKEN Nihonbashi Chuo-ku Tokyo Japan Institute of Mathematics for Industry Kyushu University Motooka Nishi-ku Fukuoka Japan and Center for Advanced Intelligence Project RIKEN Nihonbashi Chuo-ku Tokyo Japan

Kernel methods have been among the most popular techniques in machine learning, where learning tasks are solved using the property of reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel data analysis framework with reproducing kernel Hilbert C*-module (RKHM) and kernel mean embedding (KME) in RKHM. Since RKHM contains richer information than RKHS or vector-valued RKHS (vvRKHS), analysis with RKHM enables us to capture and extract structural properties in such as functional data. We show a branch of theories for RKHM to apply to data analysis, including the representer theorem, and the injectivity and universality of the proposed KME. We also show RKHM generalizes RKHS and vvRKHS. Then, we provide concrete procedures for employing RKHM and the proposed KME to data analysis.

关键词： reproducing kernel Hilbert C*-module kernel mean embedding structured data kernel PCA interaction effects

Selective inference in propensity score analysis

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ninomiya, Yoshiyuki Umezu, Yuta Takeuchi, Ichiro Department of Statistical Inference and Mathematics The Institute of Statistical Mathematics Japan School of Information and Data Science Nagasaki University Japan Department of Computer Science Nagoya Institute of Technology RIKEN Center for Advanced Intelligence Project Japan

Selective inference (post-selection inference) is a methodology that has attracted much attention in recent years in the fields of statistics and machine learning. Naive inference based on data that are also used for model selection tends to show an overestimation, and so the selective inference conditions the event that the model was selected. In this paper, we develop selective inference in propensity score analysis with a semiparametric approach, which has become a standard tool in causal inference. Specifically, for the most basic causal inference model in which the causal effect can be written as a linear sum of confounding variables, we conduct Lasso-type variable selection by adding an 1 penalty term to the loss function that gives a semiparametric estimator. Confidence intervals are then given for the coefficients of the selected confounding variables, conditional on the event of variable selection, with asymptotic guarantees. An important property of this method is that it does not require modeling of nonparametric regression functions for the outcome variables, as is usually the case with semiparametric propensity score analysis. © 2021, CC BY.

关键词： Inverse problems

Bias-Variance Reduced Local SGD for less heterogeneous federated learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Murata, Tomoya Suzuki, Taiji NTT DATA Mathematical Systems Inc. Tokyo Japan Graduate School of Information Science and Technology University of Tokyo Tokyo Japan Center for Advanced Intelligence Project RIKEN Tokyo Japan

Recently, local SGD has got much attention and been extensively studied in the distributed learning community to overcome the communication bottleneck problem. However, the superiority of local SGD to minibatch SGD only holds in quite limited situations. In this paper, we study a new local algorithm called Bias-Variance Reduced Local SGD (BVR-L-SGD) for nonconvex distributed optimization. Algorithmically, our proposed bias and variance reduced local gradient estimator fully utilizes small second-order heterogeneity of local objectives and suggests randomly picking up one of the local models instead of taking the average of them when workers are synchronized. Theoretically, under small heterogeneity of local objectives, we show that BVR-L-SGD achieves better communication complexity than both the previous non-local and local methods under mild conditions, and particularly BVR-L-SGD is the first method that breaks the barrier of communication complexity Θ(1/Ε) for general nonconvex smooth objectives when the heterogeneity is small and the local computation budget is large. Numerical results are given to verify the theoretical findings and give empirical evidence of the superiority of our method. Copyright © 2021, The Authors. All rights reserved.

关键词： Computational complexity

Koopman and Perron-Frobenius Operators on reproducing kernel Banach spaces

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Ikeda, Masahiro Ishikawa, Isao Schlosser, Corbinian LAAS-CNRS 7 avenue du colonel Roche ToulouseF-31400 France Center for Data Science Ehime University Matsuyama790-8577 Japan Center for Advanced Intelligence Project RIKEN Tokyo103-0027 Japan Department of Mathematics Keio University Yokohama223-8522 Japan

Koopman and Perron-Frobenius operators for dynamical systems have been getting popular in a number of fields in science these days. Properties of the Koopman operator essentially depend on the choice of function spaces where it acts. Particularly the case of reproducing kernel Hilbert spaces (RKHSs) draws more and more attention in data science. In this paper, we give a general framework for Koopman and Perron-Frobenius operators on reproducing kernel Banach spaces (RKBSs). More precisely, we extend basic known properties of these operators from RKHSs to RKBSs and state new results, including symmetry and sparsity concepts, on these operators on RKBS for discrete and continuous time *** Codes 47B33 Copyright © 2022, The Authors. All rights reserved.

关键词： Continuous time systems

COMPRESSION BASED BOUND FOR NON-COMPRESSED NETWORK: UNIFIED GENERALIZATION ERROR ANALYSIS OF LARGE COMPRESSIBLE DEEP NEURAL NETWORK 8

学校读者我要写书评

暂无评论

COMPRESSION BASED BOUND FOR NON-COMPRESSED NETWORK: UNIFIED ...

8th International Conference on Learning Representations, ICLR 2020

作者： Suzuki, Taiji Abe, Hiroshi Nishimura, Tomoaki Graduate School of Information Science and Technology The University of Tokyo Japan Center for Advanced Intelligence Project RIKEN Japan Japan Digital Design Japan iPride Co. Ltd. Japan NTT Data Corporation Japan

One of the biggest issues in deep learning theory is the generalization ability of networks with huge model size. The classical learning theory suggests that overparameterized models cause overfitting. However, practically used large deep models avoid overfitting, which is not well explained by the classical approaches. To resolve this issue, several attempts have been made. Among them, the compression based bound is one of the promising approaches. However, the compression based bound can be applied only to a compressed network, and it is not applicable to the non-compressed original network. In this paper, we give a unified framework that can convert compression based bounds to those for non-compressed original networks. The bound gives even better rate than the one for the compressed network by improving the bias term. By establishing the unified framework, we can obtain a data dependent generalization error bound which gives a tighter evaluation than the data independent ones. © 2020 8th International Conference on Learning Representations, ICLR 2020. All rights reserved.

关键词： Deep neural networks

SoLar: Sinkhorn Label Refinery for Imbalanced Partial-Label Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Haobo Xia, Mingxuan Li, Yixuan Mao, Yuren Feng, Lei Chen, Gang Zhao, Junbo Key Lab of Intelligent Computing based Big Data of Zhejiang Province Zhejiang University China School of Software Technology Zhejiang University China Department of Computer Sciences University of Wisconsin-Madison United States College of Computer Science Chongqing University China Center for Advanced Intelligence Project RIKEN Japan

关键词： Machine learning

Editorial: AI and IoT applications of smart buildings and smart environment design, construction and maintenance

学校读者我要写书评

暂无评论

Building and Environment 2023年 229卷

作者： Yan, Ke Zhou, Xiaokang Yang, Bin College of Mechanical and Vehicle Engineering Hunan University Changsha410012 China Department of the Built Environment College of Design and Engineering National University of Singapore 4 Architecture Drive 117566 Singapore Faculty of Data Science Shiga University Hikone5228522 Japan RIKEN Center for Advanced Intelligence Project RIKEN Tokyo1030027 Japan School of Energy and Safety Engineering Tianjin Chengjian University Tianjin300384 China

PiCO+: Contrastive Label Disambiguation for Robust Partial Label Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Haobo Xiao, Ruixuan Li, Yixuan Feng, Lei Niu, Gang Chen, Gang Zhao, Junbo Key Lab of Intelligent Computing Based Big Data of Zhejiang Province Zhejiang University Hangzhou310027 China Department of Computer Sciences University of WisconsinMadison MadisonWI53706 United States The College of Computer Science Chongqing University Chongqing400044 China RIKEN Center for Advanced Intelligence Project Tokyo Japan

Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity. Despite the promise, the performance of PLL often lags behind the supervised counterpart. In this work, we bridge the gap by addressing two key research challenges in PLL-representation learning and label disambiguation-in one coherent framework. Specifically, our proposed framework PiCO consists of a contrastive learning module along with a novel class prototype-based label disambiguation algorithm. PiCO produces closely aligned representations for examples from the same classes and facilitates label disambiguation. Theoretically, we show that these two components are mutually beneficial, and can be rigorously justified from an expectation-maximization (EM) algorithm perspective. Moreover, we study a challenging yet practical noisy partial label learning setup, where the ground-truth may not be included in the candidate set. To remedy this problem, we present an extension PiCO+ that performs distance-based clean sample selection and learns robust classifiers by a semi-supervised contrastive learning algorithm. Extensive experiments demonstrate that our proposed methods significantly outperform the current state-of-the-art approaches in standard and noisy PLL tasks and even achieve comparable results to fully supervised learning. Copyright © 2022, The Authors. All rights reserved.

关键词： Learning algorithms