检索结果-内蒙古大学图书馆

arXiv 2021年

作者： Ohno, Kentaro Kumagai, Atsutoshi NTT Computer and Data Science Laboratories

Recurrent neural networks with a gating mechanism such as an LSTM or GRU are powerful tools to model sequential data. In the mechanism, a forget gate, which was introduced to control information flow in a hidden state in the RNN, has recently been re-interpreted as a representative of the time scale of the state, i.e., a measure how long the RNN retains information on inputs. On the basis of this interpretation, several parameter initialization methods to exploit prior knowledge on temporal dependencies in data have been proposed to improve learnability. However, the interpretation relies on various unrealistic assumptions, such as that there are no inputs after a certain time point. In this work, we reconsider this interpretation of the forget gate in a more realistic setting. We first generalize the existing theory on gated RNNs so that we can consider the case where inputs are successively given. We then argue that the interpretation of a forget gate as a temporal representation is valid when the gradient of loss with respect to the state decreases exponentially as time goes back. We empirically demonstrate that existing RNNs satisfy this gradient condition at the initial training phase on several tasks, which is in good agreement with previous initialization methods. On the basis of this finding, we propose an approach to construct new RNNs that can represent a longer time scale than conventional models, which will improve the learnability for long-term sequential data. We verify the effectiveness of our method by experiments with real-world datasets. © 2021, CC BY.

关键词： Time measurement

来源：评论

学校读者我要写书评

暂无评论

Research on Transfer Learning to Give AI the Same Versatility and Skill as Humans

引用

NTT Technical Review 2021年第12期19卷 12-15页

作者： Kumagai, Atsutoshi NTT Computer and Data Science Laboratories NTT Social Informatics Laboratories Japan

Machine learning is needed to build artificial intelligence (AI), and this requires a large amount of training data. Sometimes, however, you cannot get enough high-quality training data. What’s more, to prevent an AI from becoming out of date, you need to re-train it regularly to keep pace with data that is constantly changing. In this interview, Distinguished Researcher Atsutoshi Kumagai talked to us about the technology of transfer learning, which can be used to improve AI performance even when ideal data cannot be obtained. © 2021 Nippon Telegraph and Telephone Corp.. All rights reserved.

关键词： Artificial intelligence

来源：评论

学校读者我要写书评

暂无评论

Modeling Lead-Lag Structure in Facial Expression Synchrony for Social-Psychological Outcome Prediction from Negotiation Interaction

Modeling Lead-Lag Structure in Facial Expression Synchrony f...

引用

Acoustics, Speech, and Signal Processing Workshops (ICASSPW), IEEE International Conference on

作者： Nobukatsu Hojo Saki Mizuno Satoshi Kobashikawa Ryo Masumura NTT Computer & Data Science Laboratories

This study proposes introducing facial-expression synchrony features to machine learning to estimate a customer’s psychological information from online business negotiation dialogue data. It is important for synchrony features to model the information on who led the synchrony and who followed it, the lead-lag structure, because the psychology of the leader and follower can differ. However, conventional synchrony models cannot incorporate such lead-lag structure information because they are based on the assumption that synchrony involves the co-occurrence of features in the same frame. To solve this problem, we propose using synchrony features extracted on the basis of windowed time-lagged cross-correlation, which cuts out a short segment from each of the input sequences and computes the cross-correlation between the segments. Since this method measures the similarity of signals across different frames, it is suitable for modeling the lead-lag structure. We conducted experiments based on an audio visual corpus of business negotiation dialogue assessed with various psychological measurements. The results indicate that considering lead-lag information can improve the accuracy in estimating psychological information.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Next-Speaker Prediction Based on Non-Verbal Information in Multi-Party Video Conversation

Next-Speaker Prediction Based on Non-Verbal Information in M...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Saki Mizuno Nobukatsu Hojo Satoshi Kobashikawa Ryo Masumura NTT Computer & Data Science Laboratories

We propose a method for next-speaker prediction, a task to predict who speaks in the next turn among multiple current listeners, in multi-party video conversation. Previous studies used non-verbal features, such as head movements and gaze behavior, for next-speaker prediction in face-to-face conversation. However, in video conversation, these non-verbal features are vague and ineffective because they look at the screen displaying other participants. Since non-verbal features include participant characteristics, it is necessary to use training data with rich combinations of participants to robustly predict the next speaker. Previous studies used training data with a limited number of combinations of participants because the data consist only of recorded data. Therefore, the proposed method uses 1) novel non-verbal features for next-speaker prediction in video conversation, specifically facial expressions, hand movements and speech segments, and 2) data augmentation of participant combinations in the training data. We conducted experiments to evaluate the proposed method, and the results using video-conversation data indicate its effectiveness.

关键词： Training data Oral communication Signal processing Acoustics Behavioral sciences Task analysis Speech processing

来源：评论

学校读者我要写书评

暂无评论

Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representation

Recurrent Neural Networks for Learning Long-term Temporal De...

引用

IEEE International Conference on Big Knowledge (ICBK)

作者： Kentaro Ohno Atsutoshi Kumagai NTT Computer and Data Science Laboratories

ISBN: (纸本)9781665438599

Recurrent neural networks with a gating mechanism such as an LSTM or GRU are powerful tools to model sequential data. In the mechanism, a forget gate, which was introduced to control information flow in a hidden state in the RNN, has recently been re-interpreted as a representative of the time scale of the state, i.e., a measure how long the RNN retains information on inputs. On the basis of this interpretation, several parameter initialization methods to exploit prior knowledge on temporal dependencies in data have been proposed to improve learn-ability. However, the interpretation relies on various unrealistic assumptions, such as that there are no inputs after a certain time point. In this work, we reconsider this interpretation of the forget gate in a more realistic setting. We first generalize the existing theory on gated RNNs so that we can consider the case where inputs are successively given. We then argue that the interpretation of a forget gate as a temporal representation is valid when the gradient of loss with respect to the state decreases exponentially as time goes back. We empirically demonstrate that existing RNNs satisfy this gradient condition at the initial training phase on several tasks, which is in good agreement with previous initialization methods. On the basis of this finding, we propose an approach to construct new RNNs that can represent a longer time scale than conventional models, which will improve the learnability for long-term sequential data. We verify the effectiveness of our method by experiments with real-world datasets.

关键词： Training Analytical models Recurrent neural networks Conferences Logic gates data models Time measurement

来源：评论

学校读者我要写书评

暂无评论

META-LEARNING FOR OUT-OF-DISTRIBUTION DETECTION VIA DENSITY ESTIMATION IN LATENT SPACE

arXiv

引用

arXiv 2022年

作者： Iwata, Tomoharu Kumagai, Atsutoshi NTT Communication Science Laboratories NTT Computer and Data Science Laboratories

Many neural network-based out-of-distribution (OoD) detection methods have been proposed. However, they require many training data for each target task. We propose a simple yet effective meta-learning method to detect OoD with small in-distribution data in a target task. With the proposed method, the OoD detection is performed by density estimation in a latent space. A neural network shared among all tasks is used to flexibly map instances in the original space to the latent space. The neural network is meta-learned such that the expected OoD detection performance is improved by using various tasks that are different from the target tasks. This meta-learning procedure enables us to obtain appropriate representations in the latent space for OoD detection. For density estimation, we use a Gaussian mixture model (GMM) with full covariance for each class. We can adapt the GMM parameters to in-distribution data in each task in a closed form by maximizing the likelihood. Since the closed form solution is differentiable, we can meta-learn the neural network efficiently with a stochastic gradient descent method by incorporating the solution into the meta-learning objective function. In experiments using six datasets, we demonstrate that the proposed method achieves better performance than existing meta-learning and OoD detection methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

Few-shot learning for feature selection with hilbert-schmidt independence criterion 22

Few-shot learning for feature selection with hilbert-schmidt...

引用

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Atsutoshi Kumagai Tomoharu Iwata Yasutoshi Ida Yasuhiro Fujiwara NTT Computer and Data Science Laboratories NTT Communication Science Laboratories

ISBN: (纸本)9781713871088

We propose a few-shot learning method for feature selection that can select relevant features given a small number of labeled instances. Existing methods require many labeled instances for accurate feature selection. However, sufficient instances are often unavailable. We use labeled instances in multiple related tasks to alleviate the lack of labeled instances in a target task. To measure the dependency between each feature and label, we use the Hilbert-Schmidt Independence Criterion, which is a kernel-based independence measure. By modeling the kernel functions with neural networks that take a few labeled instances in a task as input, we can encode the task- specific information to the kernels such that the kernels are appropriate for the task. Feature selection with such kernels is performed by using iterative optimization methods, in which each update step is obtained as a closed-form. This formulation enables us to directly and efficiently minimize the expected test error on features selected by a small number of labeled instances. We experimentally demonstrate that the proposed method outperforms existing feature selection methods.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Fast Binary Network Hashing via Graph Clustering

Fast Binary Network Hashing via Graph Clustering

引用

IEEE International Conference on Big data

作者： Yasuhiro Fujiwaray Masahiro Nakanoy Atsutoshi Kumagai Yasutoshi Idaz Akisato Kimura Naonori Ueda NTT Communication Science Laboratories NTT Computer and Data Science Laboratories

ISBN: (纸本)9781665480468

Network hashing converts each node of a graph into a compact binary code, and it is a useful graph analytics tool since it can reduce memory cost. INH-MF is a network hashing approach to factorize the high-order proximity matrix representing similarities between nodes. However, since it cuts small nonzero elements from the proximity matrix, it fails to effectively extract insights from the graph. Moreover, it incurs high memory and computational costs since the proximity matrix is large and dense. We propose Graph Clustering-based Network Hashing, a novel network hashing approach. To compute the proximities effectively, it uses the structural relationships between nodes and clusters obtained from a graph clustering approach. Moreover, it can efficiently compute hash codes from eigenvectors of the matrix corresponding to the graph Laplacian by using its low-rank property. Experiments show that it can more efficiently and effectively compute hash codes than previous approaches.

关键词： Laplace equations Costs Binary codes Big data Computational efficiency Matrix converters

来源：评论

学校读者我要写书评

暂无评论

Few-shot learning for unsupervised feature selection

arXiv

引用

arXiv 2021年

作者： Kumagai, Atsutoshi Iwata, Tomoharu Fujiwara, Yasuhiro NTT Computer and Data Science Laboratories NTT Communication Science Laboratories

We propose a few-shot learning method for unsupervised feature selection, which is a task to select a subset of relevant features in unlabeled data. Existing methods usually require many instances for feature selection. However, sufficient instances are often unavailable in practice. The proposed method can select a subset of relevant features in a target task given a few unlabeled target instances by training with unlabeled instances in multiple source tasks. Our model consists of a feature selector and decoder. The feature selector outputs a subset of relevant features taking a few unlabeled instances as input such that the decoder can reconstruct the original features of unseen instances from the selected ones. The feature selector uses the Concrete random variables to select features via gradient descent. To encode task-specific properties from a few unlabeled instances to the model, the Concrete random variables and decoder are modeled using permutation-invariant neural networks that take a few unlabeled instances as input. Our model is trained by minimizing the expected test reconstruction error given a few unlabeled instances that is calculated with datasets in source tasks. We experimentally demonstrate that the proposed method outperforms existing feature selection methods. Copyright © 2021, The Authors. All rights reserved.

关键词： Random variables

来源：评论

学校读者我要写书评

暂无评论

Meta-learning for relative density-ratio estimation 21

Meta-learning for relative density-ratio estimation

引用

Proceedings of the 35th International Conference on Neural Information Processing Systems

作者： Atsutoshi Kumagai Tomoharu Iwata Yasuhiro Fujiwara NTT Computer and Data Science Laboratories NTT Communication Science Laboratories

ISBN: (纸本)9781713845393

The ratio of two probability densities, called a density-ratio, is a vital quantity in machine learning. In particular, a relative density-ratio, which is a bounded extension of the density-ratio, has received much attention due to its stability and has been used in various applications such as outlier detection and dataset comparison. Existing methods for (relative) density-ratio estimation (DRE) require many instances from both densities. However, sufficient instances are often unavailable in practice. In this paper, we propose a meta-learning method for relative DRE, which estimates the relative density-ratio from a few instances by using knowledge in related datasets. Specifically, given two datasets that consist of a few instances, our model extracts the datasets' information by using neural networks and uses it to obtain instance embeddings appropriate for the relative DRE. We model the relative density-ratio by a linear model on the embedded space, whose global optimum solution can be obtained as a closed-form solution. The closed-form solution enables fast and effective adaptation to a few instances, and its differentiability enables us to train our model such that the expected test error for relative DRE can be explicitly minimized after adapting to a few instances. We empirically demonstrate the effectiveness of the proposed method by using three problems: relative DRE, dataset comparison, and outlier detection.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：