检索结果-内蒙古大学图书馆

GraphNorm: A principled approach to accelerating graph neural network training

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Cai, Tianle Luo, Shengjie Xu, keyulu He, Di Liu, Tie-Yan Wang, Liwei Department of Electrical and Computer Engineering Princeton University Zhongguancun Haihua Institute for Frontier Information Technology Key Laboratory of Machine Perception MOE School of EECS Peking University Center for Data Science Peking University Pazhou Lab Microsoft Research

Normalization is known to help the optimization of deep neural networks. Curiously, different architectures require specialized normalization methods. In this paper, we study what normalization is effective for Graph Neural Networks (GNNs). First, we adapt and evaluate the existing methods from other domains to GNNs. Faster convergence is achieved with InstanceNorm compared to BatchNorm and LayerNorm. We provide an explanation by showing that InstanceNorm serves as a preconditioner for GNNs, but such preconditioning effect is weaker with BatchNorm due to the heavy batch noise in graph datasets. Second, we show that the shift operation in InstanceNorm results in an expressiveness degradation of GNNs for highly regular graphs. We address this issue by proposing GraphNorm with a learnable shift. Empirically, GNNs with GraphNorm converge faster compared to GNNs using other normalization. GraphNorm also improves the generalization of GNNs, achieving better performance on graph classification benchmarks. Copyright © 2020, The Authors. All rights reserved.

关键词： Graph neural networks

Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Fan, Zhaoxin Zhu, Yazhi He, Yulin Sun, Qi Liu, Hongyan He, Jun Key Laboratory of Data Engineering and Knowledge Engineering of MOE School of Information Renmin University of China Beijing China No. 59 Zhongguancun Street Haidian Dist Beijing100872 China Institute of Information Science Beijing Jiaotong University No.3 Shangyuancun Haidian Dist. Beijing China School of Economics and Management Tsinghua University Haidian Dist Beijing100084 China

Object pose detection and tracking has recently attracted increasing attention due to its wide applications in many areas, such as autonomous driving, robotics, and augmented reality. Among methods for object pose detection and tracking, deep learning is the most promising one that has shown better performance than others. However, survey study about the latest development of deep learning-based methods is lacking. Therefore, this study presents a comprehensive review of recent progress in object pose detection and tracking that belongs to the deep learning technical route. To achieve a more thorough introduction, the scope of this study is limited to methods taking monocular RGB/RGBD data as input and covering three kinds of major tasks: instance-level monocular object pose detection, category-level monocular object pose detection, and monocular object pose tracking. In our work, metrics, datasets, and methods of both detection and tracking are presented in detail. Comparative results of current state-of-the-art methods on several publicly available datasets are also presented, together with insightful observations and inspiring future research directions. Copyright © 2021, The Authors. All rights reserved.

关键词： Augmented reality

Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Dong, Jianfeng Wang, Yabing Chen, Xianke Qu, Xiaoye Li, Xirong He, Yuan Wang, Xun The College of Computer and Information Engineering Zhejiang Gongshang University Hangzhou310035 China The State Key Laboratory of Information Security Institute of Information Engineering Chinese Academy of Sciences Beijing100093 China The School of Electronic Information and Communications Huazhong University of Science and Technology Hubei430074 China The Key Lab of Data Engineering and Knowledge Engineering Renmin University of China The AIMC Lab. School of Information Renmin University of China Beijing100872 China The Alibaba Group Beijing100102 China

This paper aims for the task of text-to-video retrieval, where given a query in the form of a natural-language sentence, it is asked to retrieve videos which are semantically relevant to the given query, from a great number of unlabeled videos. The success of this task depends on cross-modal representation learning that projects both videos and sentences into common spaces for semantic similarity computation. In this work, we concentrate on video representation learning, an essential component for text-to-video retrieval. Inspired by the reading strategy of humans, we propose a Reading-strategy Inspired Visual Representation Learning (RIVRL) to represent videos, which consists of two branches: a previewing branch and an intensive-reading branch. The previewing branch is designed to briefly capture the overview information of videos, while the intensive-reading branch is designed to obtain more in-depth information. Moreover, the intensive-reading branch is aware of the video overview captured by the previewing branch. Such holistic information is found to be useful for the intensive-reading branch to extract more fine-grained features. Extensive experiments on three datasets are conducted, where our model RIVRL achieves a new state-of-the-art on TGIF and VATEX. Moreover, on MSR-VTT, our model using two video features shows comparable performance to the state-of-the-art using seven video features and even outperforms models pre-trained on the large-scale HowTo100M dataset. Code is available at https://***/LiJiaBei-7/rivrl. © 2022, CC BY.

关键词： Video recording

A Markov Traffic Model for Signalized Traffic Networks Based on Bayesian Estimation

学校读者我要写书评

暂无评论

IFAC-PapersOnLine 2020年第2期53卷 15029-15034页

作者： S.Y. Liu S. Lin Y.B. Wang B. De Schutter W.H.K. Lam School of Computer Science and Technology University of Chinese Academy of Sciences Beijing 100089 China Key Lab of Big Data Mining and Knowledge Management Chinese Academy of Sciences College of Civil Engineering and Architecture Zhejiang University PR China Delft Center for Systems and Control Delft University of Technology The Netherlands Department of Civil and Structural Engineering The Hong Kong Polytechnic University Hong Kong SAR China

In order to better understand the stochastic dynamic features of signalized traffic networks, we propose a Markov traffic model to simulate the dynamics of traffic link flow density for signalized urban traffic networks with demand uncertainty. In this model, we have four different state modes for the link according to different congestion levels of the link. Each link can only be in one of the four link state modes at any time, and the transition probability from one state to the other state is estimated by Bayesian estimation based on the distributions of the dynamic traffic flow densities, and the posterior probabilities. Therefore, we use a first-order Markov Chain Model to describe the dynamics of the traffic flow evolution process. We illustrate our approach for a small traffic network. Compared with the data from the microscopic traffic simulator SUMO, the proposed model can estimate the link traffic densities accurately and can give a reliable estimation of the uncertainties in the dynamic process of signalized traffic networks.

关键词： Markov traffic model Traffic signals Bayesian Urban traffic network

Improved factor analysis algorithm in factor spaces

学校读者我要写书评

暂无评论

Improved factor analysis algorithm in factor spaces

2014 International Conference on Informatics, Networking and Intelligent Computing(INIC 2014)

作者： H.D.Wang Y.Shi P.Z.Wang H.T.Liu Key Research Lab on Big Data Mining and Knowledge Management Chinese Academy of SciencesBeijingChina Research Center on Fictitious Economy & Data ScienceChinese Academy of SciencesBeijingChina School of Mathematical SciencesUniversity of Chinese Academy of Sciences Institute of Intelligent Engineering and Mathematics Liaoning Technical University

In order to further improve the velocity and the utilization of information contained in samples,an improved version of the Factor Analysis Algorithm(FAA) in factor spaces is presented in this *** primary algorithm is considered from the whole classes during the selection of the next classified factor,by which a smaller decision domain is generated than that generated by considering from each class,and the deletion of the decision domain is critical in decreasing calculation and increasing the velocity of the ***,based on inheriting the merits of the initial algorithm,the pushing way by each column is changed into that by each class during the selection of the next classified *** change not only decreases the calculation,but also improves the utilization of the sample *** testing results also indicate that the improvement is significant and the testing accuracy rate and velocity are both better than the primary algorithm.

关键词： Factor spaces factor analysis decision domain time complexity

Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Shuai Chen, Zhengyang Lee, Kong Aik Qian, Yanmin Li, Haizhou Shenzhen Research Institute of Big Data School of Data Science The Chinese University of Hong Kong Guangdong Shenzhen518172 China Auditory Cognition and Computational Acoustics Lab The Department of Computer Science and Engineering The MoE Key Laboratory of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai200240 China Department of Electrical and Electronic Engineering The Hong Kong Polytechnic University Hong Kong

Speaker individuality information is among the most critical elements within speech signals. By thoroughly and accurately modeling this information, it can be utilized in various intelligent speech applications, such as speaker recognition, speaker diarization, speech synthesis, and target speaker extraction. In this overview, we present a comprehensive review of neural approaches to speaker representation learning from both theoretical and practical perspectives. Theoretically, we discuss speaker encoders ranging from supervised to self-supervised learning algorithms, standalone models to large pretrained models, pure speaker embedding learning to joint optimization with downstream tasks, and efforts toward interpretability. Practically, we systematically examine approaches for robustness and effectiveness, introduce and compare various open-source toolkits in the field. Through the systematic and comprehensive review of the relevant literature, research activities, and resources, we provide a clear reference for researchers in the speaker characterization and modeling field, as well as for those who wish to apply speaker modeling techniques to specific downstream tasks. Copyright © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

Promoting Business Analytics and Quantitative Management of Technology: Preface for ITQM 2016

学校读者我要写书评

暂无评论

Procedia Computer Science 2016年 91卷 1-8页

作者： Lee, Heeseok Shi, Yong Lee, Jongwon Cordova, Felisa Dzitac, Ioan Kou, Gang Li, Jaiping College of Business Korea Advanced Institute Science and Technology Dongdaemum-gu Seoul130-722 Korea Republic of Research Center on Fictitiotus Economy and Data Science Key Lab of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing100190 China College of Information Science and Technology University of Nebraska at Omaha OmahaNE68182 United States School of Business Administration Hoseo University Asan-si Chungcheongnam-do336-795 Korea Republic of Faculty of Engineering and Business University Finis Terrae Pedro de Valdivia 1509. Providencia Santiago Chile Universitatea Agora Agora University Piata Tineretului 8 Oradea410526 Romania School of Business Administration Southwestern University of Finance and Economics Chengdu Sichuan61130 China Institute of Policy and Management Chinese Academy of Sciences Beijing100190 China

Estimating Noisy Class Posterior with Part-level labels for Noisy label Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhao, Rui Shi, Bin Ruan, Jianfei Pan, Tianze Dong, Bo School of Computer Science and Technology Xi’an Jiaotong University China Shaanxi Province Key Lab of Big Data Knowledge Engineering Xi’an Jiaotong University China School of Physics Xi’an Jiaotong University China School of Distance Education Xi’an Jiaotong University China

In noisy label learning, estimating noisy class posteriors plays a fundamental role for developing consistent classifiers, as it forms the basis for estimating clean class posteriors and the transition matrix. Existing methods typically learn noisy class posteriors by training a classification model with noisy labels. However, when labels are incorrect, these models may be misled to overemphasize the feature parts that do not reflect the instance characteristics, resulting in significant errors in estimating noisy class posteriors. To address this issue, this paper proposes to augment the supervised information with part-level labels, encouraging the model to focus on and integrate richer information from various parts. Specifically, our method first partitions features into distinct parts by cropping instances, yielding part-level labels associated with these various parts. Subsequently, we introduce a novel single-to-multiple transition matrix to model the relationship between the noisy and part-level labels, which incorporates part-level labels into a classifier-consistent framework. Utilizing this framework with part-level labels, we can learn the noisy class posteriors more precisely by guiding the model to integrate information from various parts, ultimately improving the classification performance. Our method is theoretically sound, while experiments show that it is empirically effective in synthetic and real-world noisy benchmarks. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Linear Regression

学校读者我要写书评

暂无评论

arXiv 2024年

作者： He, Zelin Sun, Ying Liu, Jingyuan Li, Runze Department of Statistics Pennsylvania State University United States School of Electrical Engineering and Computer Science Pennsylvania State University United States MOE Key Laboratory of Econometrics Department of Statistics and Data Science School of Economics Wang Yanan Institute for Studies in Economics Fujian Key Lab of Statistics Xiamen University China

We consider the transfer learning problem in the high dimensional linear regression setting, where the feature dimension is larger than the sample size. To learn transferable information, which may vary across features or the source samples, we propose an adaptive transfer learning method that can detect and aggregate the feature-wise (F-AdaTrans) or sample-wise (S-AdaTrans) transferable structures. We achieve this by employing a fused-penalty, coupled with weights that can adapt according to the transferable structure. To choose the weight, we propose a theoretically informed, data-driven procedure, enabling F-AdaTrans to selectively fuse the transferable signals with the target while filtering out non-transferable signals, and S-AdaTrans to obtain the optimal combination of information transferred from each source sample. We show that, with appropriately chosen weights, F-AdaTrans achieves a convergence rate close to that of an oracle estimator with known transferable structure, and S-AdaTrans recovers existing near-minimax optimal rates as a special case. The effectiveness of the proposed method is validated using both synthetic and real data, demonstrating favorable performance compared to the existing methods. Copyright © 2024, The Authors. All rights reserved.

关键词： Transfer learning