检索结果-内蒙古大学图书馆

Shaping Rewards for Reinforcement learning with Imperfect Demonstrations using Generative Models

学校读者我要写书评

暂无评论

Shaping Rewards for Reinforcement Learning with Imperfect De...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Yuchen Wu Melissa Mozifian Florian Shkurti University of Toronto Robotics Institute University of Toronto Robotics Institute and Vector Institute Montreal Institute of Learning Algorithms (MILA) Mobile Robotics Lab (MRL) at the School of Computer Science McGill University Montréal Canada

The potential benefits of model-free reinforcement learning to real robotics systems are limited by its uninformed exploration that leads to slow convergence, lack of data-efficiency, and unnecessary interactions with the environment. To address these drawbacks we propose a method that combines reinforcement and imitation learning by shaping the reward function with a state-and-action-dependent potential that is trained from demonstration data, using a generative model. We show that this accelerates policy learning by specifying high-value areas of the state and action space that are worth exploring first. Unlike the majority of existing methods that assume optimal demonstrations and incorporate the demonstration data as hard constraints on policy optimization, we instead incorporate demonstration data as advice in the form of a reward shaping potential trained as a generative model of states and actions. In particular, we examine both normalizing flows and Generative Adversarial Networks to represent these potentials. We show that, unlike many existing approaches that incorporate demonstrations as hard constraints, our approach is unbiased even in the case of suboptimal and noisy demonstrations. We present an extensive range of simulations, as well as experiments on the Franka Emika 7DOF arm, to demonstrate the practicality of our method.

关键词： Automation Conferences Cloning Reinforcement learning Generative adversarial networks Manipulators Data models

Distantly-supervised neural relation extraction with side information using BERT

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Moreira, Johny Oliveira, Chaina MacEdo, David Zanchettin, Cleber Barbosa, Luciano Centro de Inforḿatica Universidade Federal de Pernambuco Recife Brazil Montreal Institute for Learning Algorithms University of Montreal Quebec Canada Department of Chemical and Biological Engineering Northwestern University Evanston United States

Relation extraction (RE) consists in categorizing the relationship between entities in a sentence. A recent paradigm to develop relation extractors is Distant Supervision (DS), which allows the automatic creation of new datasets by taking an alignment between a text corpus and a Knowledge Base (KB). KBs can sometimes also provide additional information to the RE task. One of the methods that adopt this strategy is the RESIDE model, which proposes a distantly-supervised neural relation extraction using side information from KBs. Considering that this method outperformed state-of-The-Art baselines, in this paper, we propose a related approach to RESIDE also using additional side information, but simplifying the sentence encoding with BERT embeddings. Through experiments, we show the effectiveness of the proposed method in Google Distant Supervision and Riedel datasets concerning the BGWA and RESIDE baseline methods. Although Area Under the Curve is decreased because of unbalanced datasets, P@N results have shown that the use of BERT as sentence encoding allows superior performance to baseline methods. Copyright © 2020, The Authors. All rights reserved.

关键词： Signal encoding

AM-MobileNet1D: A Portable Model for Speaker Recognition

学校读者我要写书评

暂无评论

AM-MobileNet1D: A Portable Model for Speaker Recognition

International Joint Conference on Neural Networks (IJCNN)

作者： João Antônio Chagas Nunes David Macêdo Cleber Zanchettin Centro de Informática Universidade Federal de Pernambuco Recife Brasil Montreal Institute for Learning Algorithms University of Montreal Canada Department of Chemical and Biological Engineering Northwestern University Evanston United States of America

ISBN: (数字)9781728169262

ISBN: (纸本)9781728169279

Speaker Recognition and Speaker Identification are challenging tasks with essential applications such as automation, authentication, and security. Deep learning approaches like SincNet and AM-SincNet presented great results on these tasks. The promising performance took these models to real-world applications that becoming fundamentally end-user driven and mostly mobile. The mobile computation requires applications with reduced storage size, non-processing and memory intensive and efficient energy-consuming. The deep learning approaches, in contrast, usually are energy expensive, demanding storage, processing power, and memory. To address this demand, we propose a portable model called Additive Margin MobileNet1D (AM-MobileNet1D) to Speaker Identification on mobile devices. We evaluated the proposed approach on TIMIT and MIT datasets obtaining equivalent or better performances concerning the baseline methods. Additionally, the proposed model takes only 11.6 megabytes on disk storage against 91.2 from SincNet and AM-SincNet architectures, making the model seven times faster, with eight times fewer parameters.

关键词： Convolution Task analysis Computational modeling Deep learning Additives Speaker recognition Computer architecture

ReSeg: A Recurrent Neural Network-Based Model for Semantic Segmentation

学校读者我要写书评

暂无评论

ReSeg: A Recurrent Neural Network-Based Model for Semantic S...

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

作者： Francesco Visin Adriana Romero Kyunghyun Cho Matteo Matteucci Marco Ciccone Kyle Kastner Yoshua Bengio Aaron Courville Montreal Institute for Learning Algorithms (MILA) University of Montreal Montreal QC Canada Dipartimento di Elettronica Informazione e Bioingegneria Politecnico di Milano Milan Italy Courant Institute and Center for Data Science New York University New York NY United States CIFAR

We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally and vertically in both directions, encoding patches or activations, and providing relevant global information. Moreover, ReNet layers are stacked on top of pre-trained convolutional layers, benefiting from generic local features. Upsampling layers follow ReNet layers to recover the original image resolution in the final predictions. The proposed ReSeg architecture is efficient, flexible and suitable for a variety of semantic segmentation tasks. We evaluate ReSeg on several widely-used semantic segmentation datasets: Weizmann Horse, Oxford Flower, and CamVid, achieving stateof-the-art performance. Results show that ReSeg can act as a suitable architecture for semantic segmentation tasks, and may have further applications in other structured prediction problems. The source code and model hyperparameters are available on https://***/fvisin/reseg.

关键词： Semantics Image segmentation Recurrent neural networks Computer architecture Image resolution Context modeling

learning What, Where and Which to Transfer

学校读者我要写书评

暂无评论

Learning What, Where and Which to Transfer

International Joint Conference on Neural Networks (IJCNN)

作者： Lucas de Lima Nogueira David Macêdo Cleber Zanchettin Fernando M. P. Neto Adriano L. I. Oliveira Centro de Informática Universidade Federal de Pernambuco Recife Brasil Montreal Institute for Learning Algorithms University of Montreal Quebec Canada Department of Chemical and Biological Engineering Northwestern University Evanston United States of America

Deep learning models often require large datasets to perform well from scratch. Transfer learning methods solve this issue by using a pre-trained source network to improve a target network training. Recent approaches involve using feature maps from the source network to guide the target network training. The latest transfer learning methods use meta-networks to enhance the knowledge transfer process. These meta-networks bridge the source and target networks, deciding which pairs of feature map layers and channels should be matched for optimal knowledge transfer. This paper improves this approach by using pixel-level information, in addition to layers and channels, for better knowledge transfer. Our experiments on multiple datasets show that the proposed approach outperforms previous baselines in scenarios with limited labels per class. The source code is available at https://***/lucasdelimanogueira/L2T-www.

关键词：

Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Sun, Fan-Yun Hoffmann, Jordan Verma, Vikas Tang, Jian National Taiwan University Mila-Quebec Institute for Learning Algorithms Canada Aalto University Finland Harvard University United States HEC Montreal Canada CIFAR AI Research Chair

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models. Copyright © 2019, The Authors. All rights reserved.

关键词： Machine learning

Distantly-Supervised Neural Relation Extraction with Side Information using BERT

学校读者我要写书评

暂无评论

Distantly-Supervised Neural Relation Extraction with Side In...

International Joint Conference on Neural Networks (IJCNN)

作者： Johny Moreira Chaina Oliveira David Macêdo Cleber Zanchettin Luciano Barbosa Centro de Informática Universidade Federal de Pernambuco Recife Brasil Montreal Institute for Learning Algorithms University of Montreal Quebec Canada Department of Chemical and Biological Engineering Northwestern University Evanston United States of America

ISBN: (数字)9781728169262

ISBN: (纸本)9781728169279

Relation extraction (RE) consists in categorizing the relationship between entities in a sentence. A recent paradigm to develop relation extractors is Distant Supervision (DS), which allows the automatic creation of new datasets by taking an alignment between a text corpus and a Knowledge Base (KB). KBs can sometimes also provide additional information to the RE task. One of the methods that adopt this strategy is the RESIDE model, which proposes a distantly-supervised neural relation extraction using side information from KBs. Considering that this method outperformed state-of-the-art baselines, in this paper, we propose a related approach to RESIDE also using additional side information, but simplifying the sentence encoding with BERT embeddings. Through experiments, we show the effectiveness of the proposed method in Google Distant Supervision and Riedel datasets concerning the BGWA and RESIDE baseline methods. Although Area Under the Curve is decreased because of unbalanced datasets, P@N results have shown that the use of BERT as sentence encoding allows superior performance to baseline methods.

关键词： Bit error rate Data mining Encoding Task analysis Syntactics Training Knowledge based systems

Count-ception: Counting by fully convolutional redundant counting

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Cohen, Joseph Paul Boucher, Geneviève Glastonbury, Craig A. Lo, Henry Z. Bengio, Yoshua Montreal Institute for Learning Algorithms Université of Montréal Harvard University Herbaria Institute for Research in Immunology and Cancer Université of Montréal Big Data Institute University of Oxford Department of Computer Science University of Massachusetts Boston

Counting objects in digital images is a process that should be replaced by machines. This tedious task is time consuming and prone to errors due to fatigue of human annotators. The goal is to have a system that takes as input an image and returns a count of the objects inside and justification for the prediction in the form of object localization. We repose a problem, originally posed by Lempitsky and Zisserman, to instead predict a count map which contains redundant counts based on the receptive field of a smaller regression network. The regression network predicts a count of the objects that exist inside this frame. By processing the image in a fully convolutional way each pixel is going to be accounted for some number of times, the number of windows which include it, which is the size of each window, (i.e., 32x32 = 1024). To recover the true count we take the average over the redundant predictions. Our contribution is redundant counting instead of predicting a density map in order to average over errors. We also propose a novel deep neural network architecture adapted from the Inception family of networks called the Count-ception network. Together our approach results in a 20% relative improvement (2.9 to 2.3 MAE) over the state of the art method by Xie, Noble, and Zisserman in 2016. Copyright © 2017, The Authors. All rights reserved.

关键词： Forecasting

Structure aware negative sampling in knowledge graphs

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Ahrabian, Kian Feizi, Aarash Salehi, Yasmin Hamilton, William L. Bose, Avishek Joey School of Computer Science McGill University Canada Department of Electrical and Computer Engineering McGill University Canada Montreal Institute of Learning Algorithms Mila Canada Canada CIFAR AI Chair Canada

learning low-dimensional representations for entities and relations in knowledge graphs using contrastive estimation represents a scalable and effective method for inferring connectivity patterns. A crucial aspect of contrastive learning approaches is the choice of corruption distribution that generates hard negative samples, which force the embedding model to learn discriminative representations and find critical characteristics of observed data. While earlier methods either employ too simple corruption distributions, i.e. uniform, yielding easy uninformative negatives or sophisticated adversarial distributions with challenging optimization schemes, they do not explicitly incorporate known graph structure resulting in suboptimal negatives. In this paper, we propose Structure Aware Negative Sampling (SANS), an inexpensive negative sampling strategy that utilizes the rich graph structure by selecting negative samples from a node’s k-hop neighborhood. Empirically, we demonstrate that SANS finds high-quality negatives that are highly competitive with SOTA methods, and requires no additional parameters nor difficult adversarial optimization. Copyright © 2020, The Authors. All rights reserved.

关键词： Crime