检索结果-内蒙古大学图书馆

Improved training of wasserstein GANs 17

学校读者我要写书评

暂无评论

Improved training of wasserstein GANs

Proceedings of the 31st International Conference on Neural Information Processing Systems

作者： Ishaan Gulrajani Faruk Ahmed Martin Arjovsky Vincent Dumoulin Aaron Courville Montreal Institute for Learning Algorithms and Google Brain Montreal Institute for Learning Algorithms Courant Institute of Mathematical Sciences Montreal Institute for Learning Algorithms and CIFAR Fellow

ISBN: (纸本)9781510860964

关键词：

Improving generative adversarial networks with denoising feature matching 5

学校读者我要写书评

暂无评论

Improving generative adversarial networks with denoising fea...

5th International Conference on learning Representations, ICLR 2017

作者： Warde-Farley, David Bengio, Yoshua Montreal Institute for Learning Algorithms Université de Montréal MontrealQC Canada CIFAR Université de Montréal MontrealQC Canada

We propose an augmented training procedure for generative adversarial networks designed to address shortcomings of the original by directing the generator towards probable configurations of abstract discriminator features. We estimate and track the distribution of these features, as computed from data, with a denoising auto-encoder, and use it to propose high-level targets for the generator. We combine this new loss with the original and evaluate the hybrid criterion on the task of unsupervised image synthesis from datasets comprising a diverse set of visual categories, noting a qualitative and quantitative improvement in the "objectness" of the resulting samples. © ICLR 2019 - Conference Track Proceedings. All rights reserved.

关键词： Generative adversarial networks

Towards an automatic turing test: learning to evaluate dialogue responses 5

学校读者我要写书评

暂无评论

Towards an automatic turing test: Learning to evaluate dialo...

5th International Conference on learning Representations, ICLR 2017

作者： Lowe, Ryan Noseworthy, Michael Serban, Iulian V. Angelard-Gontier, Nicolas Bengio, Yoshua Pineau, Joelle Reasoning and Learning Lab School of Computer Science McGill University Canada Montreal Institute for Learning Algorithms Université de Montréal Canada CIFAR

Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem. Unfortunately, existing automatic evaluation metrics are biased and correlate very poorly with human judgements of response quality (Liu et al., 2016). Yet having an accurate automatic evaluation procedure is crucial for dialogue research, as it allows rapid prototyping and testing of new models with fewer expensive human evaluations. In response to this challenge, we formulate automatic dialogue evaluation as a learning problem. We present an evaluation model (ADEM) that learns to predict human-like scores to input responses, using a new dataset of human response scores. We show that the ADEM model's predictions correlate significantly, and at level much higher than word-overlap metrics such as BLEU, with human judgements at both the utterance and system-level. We also show that ADEM can generalize to evaluating dialogue models unseen during training, an important step for automatic dialogue evaluation. © 5th International Conference on learning Representations, ICLR 2017 - Workshop Track Proceedings. All Rights Reserved.

关键词：

Improved training of wasserstein GANs

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Gulrajani, Ishaan Ahmed, Faruk Arjovsky, Martin Dumoulin, Vincent Courville, Aaron Montreal Institute for Learning Algorithms Courant Institute of Mathematical Sciences CIFAR Google Brain

Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only poor samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. We also achieve high quality generations on cifar-10 and LSUN bedrooms. Copyright © 2017, The Authors. All rights reserved.

关键词： Generative adversarial networks

Towards an automatic turing test: learning to evaluate dialogue responses

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Lowe, Ryan Noseworthy, Michael Serban, Iulian V. Gontier, Nicolas A. Bengio, Yoshua Pineau, Joelle Reasoning and Learning Lab School of Computer Science McGill University Montreal Institute for Learning Algorithms Université de Montréal Cifar Senior Fellow

Automatically evaluating the quality of dialogue responses for unstructured domains is a challenging problem. Unfortunately, existing automatic evaluation metrics are biased and correlate very poorly with human judgements of response quality. Yet having an accurate automatic evaluation procedure is crucial for dialogue research, as it allows rapid prototyping and testing of new models with fewer expensive human evaluations. In response to this challenge, we formulate automatic dialogue evaluation as a learning problem. We present an evaluation model (ADEM) that learns to predict human-like scores to input responses, using a new dataset of human response scores. We show that the ADEM model's predictions correlate significantly, and at a level much higher than word-overlap metrics such as BLEU, with human judgements at both the utterance and systemlevel. We also show that ADEM can generalize to evaluating dialogue models unseen during training, an important step for automatic dialogue evaluation. Copyright © 2017, The Authors. All rights reserved.

关键词： Quality control

A deep reinforcement learning chatbot

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Serban, Iulian V. Sankar, Chinnadhurai Germain, Mathieu Zhang, Saizheng Lin, Zhouhan Subramanian, Sandeep Kim, Taesup Pieper, Michael Chandar, Sarath Ke, Nan Rosemary Rajeshwar, Sai de Brebisson, Alexandre Sotelo, Jose M.R. Suhubdy, Dendi Michalski, Vincent Nguyen, Alexandre Pineau, Joelle Bengio, Yoshua Montreal Institute for Learning Algorithms MontrealQC Canada School of Computer Science McGill University CIFAR

We present MILABOT: a deep reinforcement learning chatbot developed by the montreal institute for learning algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including template-based models, bag-of-words models, sequence-to-sequence neural network and latent variable neural network models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than many competing systems. Due to its machine learning architecture, the system is likely to improve with additional data. Copyright © 2017, The Authors. All rights reserved.

关键词： Deep learning

INFOGRAPH: UNSUPERVISED AND SEMI-SUPERVISED GRAPH-LEVEL REPRESENTATION learning VIA MUTUAL INFORMATION MAXIMIZATION

学校读者我要写书评

暂无评论

INFOGRAPH: UNSUPERVISED AND SEMI-SUPERVISED GRAPH-LEVEL REPR...

8th International Conference on learning Representations, ICLR 2020

作者： Sun, Fan-Yun Hoffmann, Jordan Verma, Vikas Tang, Jian National Taiwan University Taiwan Mila-Quebec Institute for Learning Algorithms Canada Aalto University Finland Harvard University United States HEC Montreal Canada CIFAR AI Research

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models. © 2020 8th International Conference on learning Representations, ICLR 2020. All rights reserved.

关键词： Semantics

A deep reinforcement learning chatbot (short version)

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Serban, Iulian V. Sankar, Chinnadhurai Germain, Mathieu Zhang, Saizheng Lin, Zhouhan Subramanian, Sandeep Kim, Taesup Pieper, Michael Chandar, Sarath Ke, Nan Rosemary Rajeswar, Sai de Brebisson, Alexandre Sotelo, Jose M.R. Suhubdy, Dendi Michalski, Vincent Nguyen, Alexandre Pineau, Joelle Bengio, Yoshua Montreal Institute for Learning Algorithms MontrealQC Canada School of Computer Science McGill University CIFAR

We present MILABOT: a deep reinforcement learning chatbot developed by the montreal institute for learning algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including neural network and template-based models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than other systems. The results highlight the potential of coupling ensemble systems with deep reinforcement learning as a fruitful path for developing real-world, open-domain conversational agents. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep learning

On the impressive performance of randomly weighted encoders in summarization tasks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Pilault, Jonathan Park, Jaehong Pal, Christopher Element AI Montreal Institute for Learning Algorithms Ecole Polytechnique de Montreal Canada CIFAR AI Chair

In this work, we investigate the performance of untrained randomly initialized encoders in a general class of sequence to sequence models and compare their performance with that of fully-trained encoders on the task of abstractive summarization. We hypothesize that random projections of an input text have enough representational power to encode the hierarchical structure of sentences and semantics of documents. Using a trained decoder to produce abstractive text summaries, we empirically demonstrate that architectures with untrained randomly initialized encoders perform competitively with respect to the equivalent architectures with fully-trained encoders. We further find that the capacity of the encoder not only improves overall model generalization but also closes the performance gap between untrained randomly initialized and fulltrained encoders. To our knowledge, it is the first time that general sequence to sequence models with attention are assessed for trained and randomly projected representations on abstractive summarization. Copyright © 2020, The Authors. All rights reserved.

关键词： Signal encoding