检索结果-内蒙古大学图书馆

arXiv 2022年

作者： Luo, Shengjie Li, Shanda Zheng, Shuxin Liu, Tie-Yan Wang, Liwei He, Di Key Laboratory of Machine Perception MOE School of Intelligence Science and Technology Peking University China Machine Learning Department School of Computer Science Carnegie Mellon University United States Microsoft Research United States Center for Data Science Peking University China Zhejiang Lab China

Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far as we know, theoretical understanding of the RPE-based Transformers is largely unexplored. In this work, we mathematically analyze the power of RPE-based Transformers regarding whether the model is capable of approximating any continuous sequence-to-sequence functions. One may naturally assume the answer is in the affirmative-RPE-based Transformers are universal function approximators. However, we present a negative result by showing there exist continuous sequence-to-sequence functions that RPE-based Transformers cannot approximate no matter how deep and wide the neural network is. One key reason lies in that most RPEs are placed in the softmax attention that always generates a right stochastic matrix. This restricts the network from capturing positional information in the RPEs and limits its capacity. To overcome the problem and make the model more powerful, we first present sufficient conditions for RPE-based Transformers to achieve universal function approximation. With the theoretical guidance, we develop a novel attention module, called Universal RPE-based (URPE) Attention, which satisfies the conditions. Therefore, the corresponding URPE-based Transformers become universal function approximators. Extensive experiments covering typical architectures and tasks demonstrate that our model is parameter-efficient and can achieve superior performance to strong baselines in a wide range of applications. The code will be made publicly available at https://***/lsj2408/URPE. Copyright © 2022, The Authors. All rights reserved.

关键词： Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

Mixing Histopathology Prototypes into Robust Slide-Level Representations for Cancer Subtyping

arXiv

引用

arXiv 2023年

作者： Butke, Joshua Hashimoto, Noriaki Takeuchi, Ichiro Miyoshi, Hiroaki Ohshima, Koichi Sakuma, Jun Machine Learning and Data Mining Lab University of Tsukuba Japan RIKEN Center for Advanced Intelligence Project Japan Department of Mechanical Systems Engineering Nagoya University Japan Department of Pathology Kurume University Japan Department of Computer Science Tokyo Institute of Technology Japan

Whole-slide image analysis via the means of computational pathology often relies on processing tessellated gigapixel images with only slide-level labels available. Applying multiple instance learning-based methods or transformer models is computationally expensive as, for each image, all instances have to be processed simultaneously. The MLP-Mixer is an under-explored alternative model to common vision transformers, especially for large-scale datasets. Due to the lack of a self-attention mechanism, they have linear computational complexity to the number of input patches but achieve comparable performance on natural image datasets. We propose a combination of feature embedding and clustering to preprocess the full whole-slide image into a reduced prototype representation which can then serve as input to a suitable MLP-Mixer architecture. Our experiments on two public benchmarks and one inhouse malignant lymphoma dataset show comparable performance to current state-of-the-art methods, while achieving lower training costs in terms of computational time and memory load. Code is publicly available at https://***/butkej/ProtoMixer. Copyright © 2023, The Authors. All rights reserved.

关键词： Mixers (machinery)

来源：评论

学校读者我要写书评

暂无评论

TVEG: Model Selection of the Time-Varying Exponential Family Distributions Graphical Models

引用

IEEE Transactions on Network science and Engineering 2025年

作者： Liu, Juan Mei, Guofeng Xia, Yuanqing Wu, Xiaoqun Lu, Jinhu Beijing Forestry University School of Technology Beijing100107 China Italy Zhongyuan University of Technology Henan 451191 China Wuhan University School of Mathematics and Statistics Wuhan430072 China Beihang University State Key Laboratory of Software Development Environment Beijing Advanced Innovation Center for Big Data and Brain Machine Intelligence School of Automation Science and Electrical Engineering Beijing100191 China

The undirected graphical model, a popular class of statistical model, offers a way to describe and explain the relationships among a set of variables. However, it remains a challenge to choose a certain graphical model to explain the relationships of variables adequately, especially when the relationships of variables are rewiring over time. This paper proposes the Time-Varying Exponential Family Distributions Graphical (TVEG) models, with time-varying structures and exponential family node-wise conditional distributions. TVEG models extend the scope of available graph models and can be applied to time-varying and exponential family distribution observation data in reality. We propose the Temporally Smoothed L1regularized exponential family graphical estimator (TSLEG), an estimator to infer the structure of TVEG from observations. We derive sufficient conditions for the TSLEG to recover the block partition and sparse pattern with high probability. We derive a message-passing optimization method to solve the TSLEG for time-varying Ising, Gaussian, exponential, and Poisson graphs based on the ADMM. The synthetic network simulations corroborate the theoretical analysis. Analysing of real data of stocks and the US Senate by the time-varying exponential model and Poisson model indicates the effectiveness and practicality of TVEG models. © 2025 IEEE. All rights reserved.

关键词： ADMM exponential family distributions Graphical model pseudo maximum likelihood estimation

来源：评论

学校读者我要写书评

暂无评论

Audio Enhancement for Computer Audition – An Iterative Training Paradigm Using Sample Importance

arXiv

引用

arXiv 2024年

作者： Milling, Manuel Liu, Shuo Triantafyllopoulos, Andreas Aslan, Ilhan Schuller, Björn W. EIHW – Embedded Intelligence for Health Care & Wellbeing University of Augsburg Germany MRI Technical University of Munich Germany MCML – Munich Center for Machine Learning Germany Huawei Technologies Munich Germany MDSI – Munich Data Science Institute Germany GLAM – The Group on Language Audio & Music Imperial College London United Kingdom

Neural network models for audio tasks, such as automatic speech recognition (ASR) and acoustic scene classification (ASC), are susceptible to noise contamination for real-life applications. To improve audio quality, an enhancement module, which can be developed independently, is explicitly used at the front-end of the target audio applications. In this paper, we present an end-to-end learning solution to jointly optimise the models for audio enhancement (AE) and the subsequent applications. To guide the optimisation of the AE module towards a target application, and especially to overcome difficult samples, we make use of the sample-wise performance measure as an indication of sample importance. In experiments, we consider four representative applications to evaluate our training paradigm, i.e., ASR, speech command recognition (SCR), speech emotion recognition (SER), and ASC. These applications are associated with speech and non-speech tasks concerning semantic and non-semantic features, transient and global information, and the experimental results indicate that our proposed approach can considerably boost the noise robustness of the models, especially at low signal-to-noise ratios (SNRs), for a wide range of computer audition tasks in everyday-life noisy environments. © 2024, CC BY-NC-SA.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

A THEORY OF INITIALISATION’S IMPACT ON SPECIALISATION

arXiv

引用

arXiv 2025年

作者： Jarvis, Devon Lee, Sebastian Dominé, Clémentine Carla Juliette Saxe, Andrew M. Mannelli, Stefano Sarao School of Computer Science and Applied Mathematics University of the Witwatersrand South Africa Center for Computational Neuroscience Flatiron Institute Simons Foundation Gatsby Computational Neuroscience Unit Sainsbury Wellcome Centre UCL United Kingdom Data Science and AI Computer Science and Engineering Chalmers University of Technology University of Gothenburg Sweden Machine Intelligence and Neural Discovery Institute University of the Witwatersrand South Africa CIFAR Azrieli Global Scholar CIFAR

Prior work has demonstrated a consistent tendency in neural networks engaged in continual learning tasks, wherein intermediate task similarity results in the highest levels of catastrophic interference. This phenomenon is attributed to the network’s tendency to reuse learned features across tasks. However, this explanation heavily relies on the premise that neuron specialisation occurs, i.e. the emergence of localised representations. Our investigation challenges the validity of this assumption. Using theoretical frameworks for the analysis of neural networks, we show a strong dependence of specialisation on the initial condition. More precisely, we show that weight imbalance and high weight entropy can favour specialised solutions. We then apply these insights in the context of continual learning, first showing the emergence of a monotonic relation between task-similarity and forgetting in non-specialised networks. Finally, we show that specialization by weight imbalance is beneficial on the commonly employed elastic weight consolidation regularisation technique. Copyright © 2025, The Authors. All rights reserved.

关键词： Neural networks

来源：评论

学校读者我要写书评

暂无评论

Temporal Knowledge Graph Reasoning via Time-Distributed Representation Learning

Temporal Knowledge Graph Reasoning via Time-Distributed Repr...

引用

IEEE International Conference on data Mining (ICDM)

作者： Kangzheng Liu Feng Zhao Guandong Xu Xianzhi Wang Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China Data Science and Machine Intelligence Lab University of Technology Sydney Sydney Australia

Temporal knowledge graph (TKG) reasoning has attracted significant attention. Recent approaches for modeling historical information have led to great advances. However, the problems of time variability and unseen entities have become two major obstacles preventing further development. The time variability problem means that different historical timestamps play different roles in the inference process. Furthermore, in the context of time variability, the unseen entity problem means that a query cannot obtain a predicted entity that is unseen in the scale-varying history rather than in a fixed set, thus turning from static to dynamic. In this paper, we propose a novel method named DHU-NET for addressing the time variability challenge and the dynamic unseen entity challenge derived from it. With regard to the former concern, we propose a time-distributed representation learning method based on a graph convolutional network(GCN) and a self-attention mechanism, which learns the distributed representations of facts at different historical timestamps and comprehensively pays different levels of attention to the different timestamps. With regard to the latter issue, we extract the unseen entities from a global static KG based on a copy mechanism and bring them into consideration during the final prediction step. Experiments on six benchmark datasets demonstrate the substantial improvements achieved by DHUNET in terms of multiple evaluation metrics. Our released codes are available at https://***/CGCL-codes/DHUNET.

关键词： Representation learning Measurement Knowledge engineering Predictive models Turning Cognition data models

来源：评论

学校读者我要写书评

暂无评论

Cross-Domain Animal Pose Estimation with Skeleton Anomaly-Aware Learning

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Han, Le Chen, Kaixuan Zhao, Lei Jiang, Yangbo Wang, Pengfei Zheng, Nenggan Zhejiang Hangzhou310007 China Zhejiang University College of Computer Science and Technology Zhejiang Hangzhou310007 China Zhejiang University State Key Laboratory of Blockchain and Data Security Zhejiang Hangzhou310007 China Institute of Blockchain and Data Security. China Zhejiang University School of Software Technology Ningbo China Zhejiang University State Key Lab of Brain-Machine Intelligence Hangzhou310007 China Zhejiang Provincial Government ZJU Collaborative Innovation Center for Artificial Intelligence by MOE Hangzhou310007 China Bengbu University School of Computer and Information Engineering Bengbu233030 China

Animal pose estimation is often constrained by the scarcity of annotations and the diversity of scenarios and species. The pseudo-label generation based unsupervised domain adaptation paradigm, which discriminates the predicted keypoints of unlabeled data based on the skeleton position consistency, has demonstrated effectiveness for such problems. However, existing methods generate pseudo-labels with massive false positives, because they cannot effectively distinguish sample pairs with the same errors. In this study, we propose a cross-domain animal pose estimation model from a novel perspective of skeleton anomaly learning. We construct a graph contrastive learning mechanism to acquire the skeleton anomaly-aware knowledge, which enables the generation of accurate pseudo-labels for target domain and imposes graph constraint on unlabeled data. And a skeleton anomaly-feedback based domain adaptation framework is designed to facilitate implicit alignment of object-specific features and joint training of cross-domain. Besides, we propose a novel rat pose dataset named UDARP-9.4K to address the gap of small-sized animal pose datasets encompassing diverse experimental scenarios. The related datasets are reviewed and evaluated in detail. Extensive experiments are conducted on UDARP-9.4K and two public datasets to demonstrate the superiority of the proposed model in cross-scenarios and cross-species animal pose estimation tasks. Further analysis reveals the effectiveness of the proposed model for skeleton structure feature learning. The UDARP-9.4K dataset is available here. © 1991-2012 IEEE.

关键词： Invertebrates

来源：评论

学校读者我要写书评

暂无评论

Homophily Induced Contrastive Attributed Graph Clustering

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Chen, Man-Sheng Lai, Pei-Yuan Liao, De-Zhang Wang, Chang-Dong Lai, Jian-Huang Sun Yat-sen University School of Computer Science and Engineering Guangzhou China Guangxi Zhuang Autonomous Region Information Center Guangxi Key Laboratory of Digital Infrastructure Nanning China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China South China Technology Commercialization Center Guangzhou China Guangdong Key Laboratory of Information Security Technology Guangzhou China

Attributed graph clustering, aiming to discover the underlying graph structure and partition the graph nodes into several disjoint categories, is a basic task in graph data analysis. Although recent efforts over graph contrastive clustering have achieved decent performances, most of them get accustomed to construct the positive neighbor set by the generated pseudo clustering information, directly ignoring the ready-made neighbor nodes and the underlying semantics of edges in a graph. How to well deal with the graph neighbor-specific information to facilitate the performance of graph contrastive clustering is still a challenging problem. Therefore, in this work, we propose a novel Homophily Induced Contrastive Attributed Graph Clustering (HomoCAGC) method, where the power of homophily is exploited in facilitating the performance of contrastive attributed graph clustering, while the pseudo homophily in a graph is also explored and distinguished. Especially, the node feature as reliable information guidance is used to compute the underlying feature-oriented pair-wise node similarity, based on which the positive node pairs in contrastive regularizer are adjusted for better node representation characterization. According to the refined node representations, a triplet self-supervised clustering objective is well-designed to ensure the output embedding is cluster-oriented, and suitable for the downstream clustering task. Extensive experiments on seven benchmarks are conducted to demonstrate the effectiveness of HomoCAGC. © 1991-2012 IEEE.

关键词： Attributed graph clustering contrastive learning graph homophily self-supervised clustering

来源：评论

学校读者我要写书评

暂无评论

An Automatic Analysis of Ultrasound Vocalisations for the Prediction of Interaction Context in Captive Egyptian Fruit Bats

An Automatic Analysis of Ultrasound Vocalisations for the Pr...

引用

European Signal Processing Conference (EUSIPCO)

作者： Andreas Triantafyllopoulos Alexander Gebhard Manuel Milling Simon Rampp Björn Schuller CHI - Chair of Health Informatics Technical University of Munich MRI Munich Germany EIHW - Chair of Embedded Intelligence for Health Care and Wellbeing Augsburg Germany MCML - Munich Center for Machine Learning Munich Germany MDSI - Munich Data Science Institute Munich Germany GLAM - Group on Language Audio & Music Imperial College London UK

ISBN: (数字)9789464593617

ISBN: (纸本)9798331519773

Prior work in computational bioacoustics has mostly focused on the detection of animal presence in a particular habitat. However, animal sounds contain much richer information than mere presence; among others, they encapsulate the interactions of those animals with other members of their species. Studying these interactions is almost impossible in a naturalistic setting, as the ground truth is often lacking. The use of animals in captivity instead offers a viable alternative pathway. However, most prior works follow a traditional, statistics-based approach to analysing interactions. In the present work, we go beyond this standard framework by attempting to predict the underlying context in interactions between captive Rousettus Aegyptiacus using deep neural networks. We reach an unweighted average recall of over 30% - more than thrice the chance level - and show error patterns that differ from our statistical analysis. This work thus represents an important step towards the automatic analysis of states in animals from sound.

关键词： Ultrasonic imaging Animals Statistical analysis Habitats Europe Signal processing Epidemiology Biomedical monitoring Standards Monitoring

来源：评论

学校读者我要写书评

暂无评论

Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?

arXiv

引用

arXiv 2022年

作者： Wang, Chuwei Li, Shanda He, Di Wang, Liwei School of Mathematical Sciences Peking University China Machine Learning Department School of Computer Science Carnegie Mellon University United States National Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University China Center for Data Science Peking University China Zhejiang Lab China

The Physics-Informed Neural Network (PINN) approach is a new and promising way to solve partial differential equations using deep learning. The L2 Physics-Informed Loss is the de-facto standard in training Physics-Informed Neural Networks. In this paper, we challenge this common practice by investigating the relationship between the loss function and the approximation quality of the learned solution. In particular, we leverage the concept of stability in the literature of partial differential equation to study the asymptotic behavior of the learned solution as the loss approaches zero. With this concept, we study an important class of high-dimensional non-linear PDEs in optimal control, the Hamilton-Jacobi-Bellman (HJB) Equation, and prove that for general Lp Physics-Informed Loss, a wide class of HJB equation is stable only if p is sufficiently large. Therefore, the commonly used L2 loss is not suitable for training PINN on those equations, while L∞ loss is a better choice. Based on the theoretical insight, we develop a novel PINN training algorithm to minimize the L∞ loss for HJB equations which is in a similar spirit to adversarial training. The effectiveness of the proposed algorithm is empirically demonstrated through experiments. Our code is released at https://***/LithiumDA/L_inf-PINN. Copyright © 2022, The Authors. All rights reserved.

关键词： Partial differential equations

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：