检索结果-内蒙古大学图书馆

CUTE: Measuring LLMs' Understanding of Their Tokens

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Edman, Lukas Schmid, Helmut Fraser, Alexander Center for Information and Language Processing LMU Munich Germany School of Computation Information and Technology TU Munich Germany Munich Center for Machine Learning Germany Munich Data Science Institute Germany

Large Language Models (LLMs) show remarkable performance on a wide variety of tasks. Most LLMs split text into multi-character tokens and process them as atomic units without direct access to individual characters. This raises the question: To what extent can LLMs learn orthographic information? To answer this, we propose a new benchmark, CUTE, which features a collection of tasks designed to test the orthographic knowledge of LLMs. We evaluate popular LLMs on CUTE, finding that most of them seem to know the spelling of their tokens, yet fail to use this information effectively to manipulate text, calling into question how much of this knowledge is generalizable. Copyright © 2024, The Authors. All rights reserved.

关键词：

BIM: Improving Graph Neural Networks with Balanced Influence Maximization 40

学校读者我要写书评

暂无评论

BIM: Improving Graph Neural Networks with Balanced Influence...

40th IEEE International Conference on data Engineering, ICDE 2024

作者： Zhang, Wentao Gao, Xinyi Yang, Ling Cao, Meng Huang, Ping Shan, Jiulong Yin, Hongzhi Cui, Bin Peking University Center for Machine Learning Research China Institute of Advanced Algorithms Research Shanghai China National Engineering Labratory for Big Data Analytics and Applications The University of Queensland Australia Peking University Key Lab of High Confidence Software Technologies China Apple Inc. Institute of Computational Social Science Peking University Qingdao China

ISBN: (纸本)9798350317152

The imbalanced data classification problem has aroused lots of concerns from both academia and industry since data imbalance is a widespread phenomenon in many real-world scenarios. Although this problem has been well researched from the view of imbalanced class samples, we further argue that graph neural networks (GNNs) expose a unique source of imbalance from the influenced nodes of different classes of labeled nodes, i.e., labeled nodes are imbalanced in terms of the number of nodes they influenced during the influence propagation in GNNs. To tackle this previously unexplored influence-imbalance issue, we connect social influence maximization with the imbalanced node classification problem and propose balanced influence maximization (BIM). Specifically, BIM greedily assigns the pseudo label to the node which can maximize the number of influenced nodes in GNN training while making the influence of each class more balance. Experimental results on five public datasets demonstrate the effectiveness of our method in relieving the influence-imbalance issue. For example, when training a GCN with an imbalance ratio of 0.1, BIM significantly outperforms the most competitive baseline by 0.6% -9.8% in five public datasets in terms of the F1 score. © 2024 IEEE.

关键词： Graph neural networks

RDMP: A Reference Detection and Mask Propagation Pipeline for Numerous 3D Glomeruli Segmentation in Large Volumes 22

学校读者我要写书评

暂无评论

RDMP: A Reference Detection and Mask Propagation Pipeline fo...

22nd IEEE International Symposium on Biomedical Imaging, ISBI 2025

作者： Lin, Ziling Li, Zehua Chen, Zifan Lu, Yao Zhou, Fangxu Dong, Bin Zhang, Li Li, Haifeng Peking University Center for Data Science Beijing China Peking University First Hospital Beijing China College of Future Technology Peking University Beijing China Peking University Beijing International Center for Mathematical Research Beijing China Peking University Center for Machine Learning Research Beijing China Peking University National Biomedical Imaging Center Beijing China

ISBN: (纸本)9798331520526

Quantitative analysis of numerous repetitive structures in large volumes is one of the most challenging tasks in biomedical imaging, primarily due to its intensive manual requirements. Kidney glomeruli exemplify this challenge due to their dispersion across the cortical region, high density and microscopic size. To address this, we present a Reference Detection and Mask Propagation (RDMP) pipeline that combines object detection with 2D and 3D mask propagation, gradually achieving efficient segmentation of the entire volume by using a 2D reference example. Our method demonstrates robust performance, achieving a Dice Similarity Coefficient (DSC) of 0.9096 on microCT data, and when generalized to Synchrotron Radiation X-ray (SRX) data, reaching a DSC of 0.8431 with only minimal bounding box fine-tuning. Overall, this pipeline shows strong potential for efficient, large-scale kidney analysis, significantly reducing labor demands and accelerating kidney disease research. © 2025 IEEE.

关键词： Image segmentation

Bounding the number of reticulation events for displaying multiple trees in a phylogenetic network

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Yufeng Zhang, Louxin School of Computing University of Connecticut StorrsCT06269 United States Department of Mathematics Center for Data Science and Machine Learning National University of Singapore Singapore119076 Singapore

Reconstructing a parsimonious phylogenetic network that displays multiple phylogenetic trees is an important problem in phylogenetics, where the complexity of the inferred networks is measured by reticulation numbers. The reticulation number for a set of trees is defined as the minimum number of reticulations in a phylogenetic network that displays those trees. A mathematical problem is bounding the reticulation number for multiple trees over a fixed number of taxa. While this problem has been extensively studied for two trees, much less is known about the upper bounds on the reticulation numbers for three or more arbitrary trees. In this paper, we present a few non-trivial upper bounds on reticulation numbers for three or more *** Codes 5C30 © 2024, CC BY.

关键词： Abiotic

A Minimax Optimal Control Approach for Robust Neural ODEs

学校读者我要写书评

暂无评论

A Minimax Optimal Control Approach for Robust Neural ODEs

European Control Conference (ECC)

作者： Cristina Cipriani Alessandro Scagliotti Tobias Wöhrer Department of Mathematics Technical University Munich (TUM) Munich Germany Munich Data Science Institute (MDSI) Munich Germany Munich Center for Machine Learning (MCML) Munich Germany

ISBN: (数字)9783907144107

ISBN: (纸本)9798331540920

In this paper, we address the adversarial training of neural ODEs from a robust control perspective. This is an alternative to the classical training via empirical risk minimization, and it is widely used to enforce reliable outcomes for input perturbations. Neural ODEs allow the interpretation of deep neural networks as discretizations of control systems, unlocking powerful tools from control theory for the development and the understanding of machine learning. In this specific case, we formulate the adversarial training with perturbed data as a minimax optimal control problem, for which we derive first order optimality conditions in the form of Pontryagin's Maximum Principle. We provide a novel interpretation of robust training leading to an alternative weighted technique, which we test on a low-dimensional classification task.

关键词： Training Weight measurement Robust control Risk minimization Perturbation methods Optimal control Artificial neural networks

A REDUCTION-BASED FRAMEWORK FOR CONSERVATIVE BANDITS AND REINFORCEMENT learning 10

学校读者我要写书评

暂无评论

A REDUCTION-BASED FRAMEWORK FOR CONSERVATIVE BANDITS AND REI...

10th International Conference on learning Representations, ICLR 2022

作者： Yang, Yunchang Wu, Tianhao Zhong, Han Garcelon, Evrard Pirotta, Matteo Lazaric, Alessandro Wang, Liwei Du, Simon S. Center for Data Science Peking University China University of California Berkeley United States Facebook AI Research Key Laboratory of Machine Perception MOE School of Artificial Intelligence Peking University China International Center for Machine Learning Research Peking University China University of Washington United States

We study bandits and reinforcement learning (RL) subject to a conservative constraint where the agent is asked to perform at least as well as a given baseline policy. This setting is particular relevant in real-world domains including digital marketing, healthcare, production, finance, etc. In this paper, we present a reduction-based framework for conservative bandits and RL, in which our core technique is to calculate the necessary and sufficient budget obtained from running the baseline policy. For lower bounds, we improve the existing lower bound for conservative multi-armed bandits and obtain new lower bounds for conservative linear bandits, tabular RL and low-rank MDP, through a black-box reduction that turns a certain lower bound in the nonconservative setting into a new lower bound in the conservative setting. For upper bounds, in multi-armed bandits, linear bandits and tabular RL, our new upper bounds tighten or match existing ones with significantly simpler analyses. We also obtain a new upper bound for conservative low-rank MDP. © 2022 ICLR 2022 - 10th International Conference on learning Representationss. All rights reserved.

关键词： Reinforcement learning

Decoupling Common and Unique Representations for Multimodal Self-supervised learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Yi Albrecht, Conrad M. Braham, Nassim Ait Ali Liu, Chenying Xiong, Zhitong Zhu, Xiao Xiang Data Science in Earth Observation Technical University of Munich Germany Remote Sensing Technology Institute German Aerospace Center Germany Munich Center for Machine Learning Germany

The increasing availability of multi-sensor data sparks wide interest in multimodal self-supervised learning. However, most existing approaches learn only common representations across modalities while ignoring intra-modal training and modality-unique representations. We propose Decoupling Common and Unique Representations (DeCUR), a simple yet effective method for multimodal self-supervised learning. By distinguishing inter- and intra-modal embeddings through multimodal redundancy reduction, DeCUR can integrate complementary information across different modalities. We evaluate DeCUR in three common multimodal scenarios (radar-optical, RGB-elevation, and RGB-depth), and demonstrate its consistent improvement regardless of architectures and for both multimodal and modality-missing settings. With thorough experiments and comprehensive analysis, we hope this work can provide valuable insights and raise more interest in researching the hidden relationships of multimodal representations4 Copyright © 2023, The Authors. All rights reserved.

关键词： Self-supervised learning

Optimal bounds for p sensitivity sampling via 2 augmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Munteanu, Alexander Omlor, Simon Dortmund Data Science Center Faculties of Statistics and Computer Science TU Dortmund University Dortmund Germany Faculty of Statistics and Lamarr Institute for Machine Learning and Artificial Intelligence TU Dortmund University Dortmund Germany

data subsampling is one of the most natural methods to approximate a massively large data set by a small representative proxy. In particular, sensitivity sampling received a lot of attention, which samples points proportional to an individual importance measure called sensitivity. This framework reduces in very general settings the size of data to roughly the VC dimension d times the total sensitivity S while providing strong (1 ± Ε) guarantees on the quality of approximation. The recent work of Woodruff and Yasuda (2023c) improved substantially over the general Õ(Ε−2Sd) bound for the important problem of p subspace embeddings to Õ(Ε−2S2/p) for p ∈ [1, 2]. Their result was subsumed by an earlier Õ(Ε−2Sd1−p/2) bound which was implicitly given in the work of Chen and Derezinski (2021). We show that their result is tight when sampling according to plain p sensitivities. We observe that by augmenting the p sensitivities by 2 sensitivities, we obtain better bounds improving over the aforementioned results to optimal linear Õ(Ε−2(S + d)) = Õ(Ε−2d) sampling complexity for all p ∈ [1, 2]. In particular, this resolves an open question of Woodruff and Yasuda (2023c) in the affirmative for p ∈ [1, 2] and brings sensitivity subsampling into the regime that was previously only known to be possible using Lewis weights (Cohen and Peng, 2015). As an application of our main result, we also obtain an Õ(Ε−2µd) sensitivity sampling bound for logistic regression, where µ is a natural complexity measure for this problem. This improves over the previous Õ(Ε−2µ2d) bound of Mai et al. (2021) which was based on Lewis weights subsampling. Copyright © 2024, The Authors. All rights reserved.

关键词： Importance sampling

Turnstile p leverage score sampling with applications

学校读者我要写书评

暂无评论

arXiv 2024年

The turnstile data stream model offers the most flexible framework where data can be manipulated dynamically, i.e., rows, columns, and even single entries of an input matrix can be added, deleted, or updated multiple times in a data stream. We develop a novel algorithm for sampling rows ai of a matrix A ∈ n×d, proportional to their p norm, when A is presented in a turnstile data stream. Our algorithm not only returns the set of sampled row indexes, it also returns slightly perturbed rows ãi ≈ ai, and approximates their sampling probabilities up to Ε relative error. When combined with preconditioning techniques, our algorithm extends to p leverage score sampling over turnstile data streams. With these properties in place, it allows us to simulate subsampling constructions of coresets for important regression problems to operate over turnstile data streams with very little overhead compared to their respective offline subsampling algorithms. For logistic regression, our framework yields the first algorithm that achieves a (1 + Ε) approximation and works in a turnstile data stream using polynomial sketch/subsample size, improving over O(1) approximations, or exp(1/Ε) sketch size of previous work. We compare experimentally to plain oblivious sketching and plain leverage score sampling algorithms for p and logistic regression. Copyright © 2024, The Authors. All rights reserved.

关键词： Approximation algorithms