检索结果-内蒙古大学图书馆

A contrastive framework for neural text generation 22

学校读者我要写书评

暂无评论

A contrastive framework for neural text generation

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Yixuan Su Tian Lan Yan Wang Dani Yogatama Lingpeng Kong Nigel Collier Language Technology Lab University of Cambridge Tencent AI Lab DeepMind Department of Computer Science The University of Hong Kong

ISBN: (纸本)9781713871088

Text generation is of great importance to many natural language processing applications. However, maximization-based decoding methods (e.g., beam search) of neural language models often lead to degenerate solutions—the generated text is unnatural and contains undesirable repetitions. Existing approaches introduce stochasticity via sampling or modify training objectives to decrease the probabilities of certain tokens (e.g., unlikelihood training). However, they often lead to solutions that lack coherence. In this work, we show that an underlying reason for model degeneration is the anisotropic distribution of token representations. We present a contrastive solution: (i) SimCTG, a contrastive training objective to calibrate the model's representation space, and (ii) a decoding method—contrastive search—to encourage diversity while maintaining coherence in the generated text. Extensive experiments and analyses on three benchmarks from two languages demonstrate that our proposed approach significantly outperforms current state-of-the-art text generation methods as evaluated by both human and automatic metrics.

关键词：

Fine-Grained Distribution-Dependent Learning Curves

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Bousquet, Olivier Hanneke, Steve Moran, Shay Shafer, Jonathan Tolstikhin, Ilya Google Brain Team United States Computer Science Department Purdue University United States Department of Mathematics Department of Computer Science Technion - Israel Institute of Technology Google Research United States Computer Science Division UC Berkeley United States

Learning curves plot the expected error of a learning algorithm as a function of the number of labeled samples it receives from a target distribution. They are widely used as a measure of an algorithm's performance, but classic PAC learning theory cannot explain their behavior. As observed by Antos and Lugosi (1996, 1998), the classic 'No Free Lunch' lower bounds only trace the upper envelope above all learning curves of specific target distributions. For a concept class with VC dimension d the classic bound decays like d/n, yet it is possible that the learning curve for every specific distribution decays exponentially. In this case, for each n there exists a different 'hard' distribution requiring d/n samples. Antos and Lugosi asked which concept classes admit a 'strong minimax lower bound' - a lower bound of d0/n that holds for a fixed distribution for infinitely many n. We solve this problem in a principled manner, by introducing a combinatorial dimension called VCL that characterizes the best d0 for which d0/n is a strong minimax lower bound. Our characterization strengthens the lower bounds of Bousquet, Hanneke, Moran, van Handel, and Yehudayoff (2021), and it refines their theory of learning curves, by showing that for classes with finite VCL the learning rate can be decomposed into a linear component that depends only on the hypothesis class and an exponential component that depends also on the target distribution. As a corollary, we recover the lower bound of Antos and Lugosi (1996, 1998) for half-spaces in Rd. Finally, to provide another viewpoint on our work and how it compares to traditional PAC learning bounds, we also present an alternative formulation of our results in a language that is closer to the PAC setting. © 2022, CC BY-NC-ND.

关键词： Learning algorithms

Combating Toxicity, Harassment, and Abuse in Online Social Spaces: A Workshop at CHI 2023

学校读者我要写书评

暂无评论

Combating Toxicity, Harassment, and Abuse in Online Social S...

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA 2023

作者： Mandryk, Regan L. Frommel, Julian Goyal, Nitesh Freeman, Guo Lampe, Cliff Vieweg, Sarah Wohn, Donghee Yvette Department of Computer Science University of Saskatchewan Canada Information and Computing Sciences Utrecht University Netherlands Google United States School of Computing Clemson University United States School of Information University of Michigan United States Cash App United States New Jersey Institute of Technology United States

ISBN: (纸本)9781450394222

Online social spaces provide much needed connection and belonging - particularly in a context of continued lack of global mobility due to the ongoing Covid-19 pandemic and climate crisis. However, the norms of online social spaces can create environments in which toxic behaviour is normalized, tolerated or even celebrated. This can occur without consequence, leaving its members vulnerable to hate, harassment, and abuse. A vast majority of adults have experienced toxicity online and the harm is even more prevalent for members of marginalized and minoritized groups, who are more often the targets of online abuse. Although there is significant work on toxicity in the SIGCHI community, approaches and knowledge have typically been siloed by the domain of investigation (e.g., social media, multiplayer games, social VR). We argue that cross-disciplinary efforts will benefit not only the various communities and situations in which abuse occurs, but that bringing together researchers from different backgrounds and specialties will provide a robust and rich understanding of how to tackle online toxicity at scale. © 2023 Owner/Author.

关键词： Toxicity

Utilizing Deep Convolutional Neural Network and Local Binary Pattern Fusion for Plant Leaf Disease Identification: An Integrated Approach

学校读者我要写书评

暂无评论

Utilizing Deep Convolutional Neural Network and Local Binary...

Industrial Electronics: Developments & Applications (ICIDeA), IEEE International Conference on-

作者： S. Rahamat Basha Pratibha C. Kaladeep Yalagi Balaram Puli Pandian Sundaramoorthy Rajesh Daruvuri S Vinayagapriya Department of computer science and engineering Malla reddy college of engineering and technology Department of Computer Science and Engineering Walchand Institute of Technology Solapur Pratibhayalagi Engineering and Data Science Everest Computers Inc Application Developer EL CIC-1W-AMI IBM Technical Account Manager Cloud Data and AI Google Dept of ece St.josephs college of engineering

ISBN: (数字)9798331533205

ISBN: (纸本)9798331533212

Precise detection of plant diseases is essential in agriculture to maximize crop management and yield. The present study presents an innovative approach that integrates convolutional neural networks (CNNs) with local binary patterns (LBPs) in order to enhance the categorization of plant leaf diseases. The intricacy of plant disease images complicates the evaluation of their many features by conventional approaches. To solve this, the suggested technique utilizes the strengths of both CNNs and LBPs. The CNN component is highly effective in extracting discriminative features from raw pictures using hierarchical representations, which are crucial for differentiating between different illness categories. Simultaneously, the LBP descriptor enhances its usefulness by acquiring intricate texture information from specific areas of the images, so providing more effectiveness in distinguishing small differences resulting from various diseases. Utilizing a feature fusion approach, the merging of CNN and LBP features merges spatial and textural data, resulting in a more thorough representation of the input images. The fusion described herein significantly improves the precision of plant disease classification. The analysis of benchmark datasets of plant leaf diseases showed that this integrated method outperforms the use of CNNs or LBPs individually in terms of classification accuracy. The research findings indicate that the suggested approach, which combines deep Convolutional Neural Networks (CNNs) with Linear Bayesian Processes (LBPs), provides a more efficient approach for detecting plant leaf illnesses. This, in turn, makes a substantial contribution to crop management and agricultural productivity.

关键词： Productivity Plant diseases Accuracy Merging Crops Benchmark testing Feature extraction Spatial databases Bayes methods Convolutional neural networks

Utilizing Deep Convolutional Neural Network and Local Binary Pattern Fusion for Plant Leaf Disease Identification: An Integrated Approach 3

学校读者我要写书评

暂无评论

Utilizing Deep Convolutional Neural Network and Local Binary...

3rd IEEE International Conference on Industrial Electronics: Developments and Applications, ICIDeA 2025

作者： Rahamat Basha, S. Kaladeep Yalagi, Pratibha C. Puli, Balaram Sundaramoorthy, Pandian Daruvuri, Rajesh Vinayagapriya, S. Malla reddy college of engineering and technology Department of computer science and engineering India Walchand Institute of Technology Solapur Department of Computer Science and Engineering Solapur India Engineering and Data Science Everest Computers Inc India Application Developer EL CIC-1W-AMI IBM India Google India St.josephs college of engineering Dept of ece India

ISBN: (纸本)9798331533205

关键词： Convolutional neural networks

DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Pandey, Kushagra Mukherjee, Avideep Rai, Piyush Kumar, Abhishek Department of Computer Science University of California Irvine United States Department of Computer Science Indian Institute of Technology Kanpur India Google Research Brain Team

Diffusion probabilistic models have been shown to generate state-of-the-art results on several competitive image synthesis benchmarks but lack a low-dimensional, interpretable latent space, and are slow at generation. On the other hand, standard Variational Autoencoders (VAEs) typically have access to a low-dimensional latent space but exhibit poor sample quality. We present DiffuseVAE, a novel generative framework that integrates VAE within a diffusion model framework, and leverage this to design novel conditional parameterizations for diffusion models. We show that the resulting model equips diffusion models with a low-dimensional VAE inferred latent code which can be used for downstream tasks like controllable synthesis. The proposed method also improves upon the speed vs quality tradeoff exhibited in standard unconditional DDPM/DDIM models (for instance, FID of 16.47 vs 34.36 using a standard DDIM on the CelebA-HQ-128 benchmark using T=10 reverse process steps) without having explicitly trained for such an objective. Furthermore, the proposed model exhibits synthesis quality comparable to state-of-the-art models on standard image synthesis benchmarks like CIFAR-10 and CelebA-64 while outperforming most existing VAE-based methods. Lastly, we show that the proposed method exhibits inherent generalization to different types of noise in the conditioning signal. For reproducibility, our source code is publicly available at https://***/kpandey008/DiffuseVAE. © 2022, CC BY-NC-SA.

关键词： Diffusion

The Clever Hans Effect in Unsupervised Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Kauffmann, Jacob Dippel, Jonas Ruff, Lukas Samek, Wojciech Müller, Klaus-Robert Montavon, Grégoire Department of Electrical Engineering and Computer Science Technische Universität Berlin Germany BIFOLD – Berlin Institute for the Foundations of Learning and Data Berlin Germany Aignostics Berlin Germany Department of Mathematics and Computer Science Freie Universität Berlin Germany Department of Artificial Intelligence Fraunhofer HHI Berlin Germany Department of Artificial Intelligence Korea University Seoul Korea Republic of Max-Planck Institute for Informatics Saarbrücken Germany Google Deepmind Berlin Germany

Unsupervised learning has become an essential building block of AI systems. The representations it produces, e.g. in foundation models, are critical to a wide variety of downstream applications. It is therefore important to carefully examine unsupervised models to ensure not only that they produce accurate predictions, but also that these predictions are not "right for the wrong reasons", the so-called Clever Hans (CH) effect. Using specially developed Explainable AI techniques, we show for the first time that CH effects are widespread in unsupervised learning. Our empirical findings are enriched by theoretical insights, which interestingly point to inductive biases in the unsupervised learning machine as a primary source of CH effects. Overall, our work sheds light on unexplored risks associated with practical applications of unsupervised learning and suggests ways to make unsupervised learning more robust. Copyright © 2024, The Authors. All rights reserved.

关键词： Adversarial machine learning

Lightweight, Multi-Speaker, Multi-Lingual Indic Text-to-Speech

学校读者我要写书评

暂无评论

Lightweight, Multi-Speaker, Multi-Lingual Indic Text-to-Spee...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Abhayjeet Singh Amala Nagireddi Deekshitha G Jesuraja Bandekar Roopa R Sandhya Badiger Sathvik Udupa Prasanta Kumar Ghosh Hema A Murthy Heiga Zen Pranaw Kumar Kamal Kant Amol Bole Bira Chandra Singh Keiichi Tokuda Mark Hasegawa-Johnson Philipp Olbrich Electrical Engineering Department Indian Institute of Science (IISc) Bangalore India Department of Computer Science & Engineering Indian Institute of Technology Madras India Google Japan CDAC Mumbai India Department of Computer Science Nagoya Institute of Technology Japan Department of Electrical and Computer Engineering University of Illinois Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH Bonn Germany

ISBN: (数字)9781728163277

ISBN: (纸本)9781728163284

The Lightweight, Multi-speaker, Multi-lingual Indic Text-to-Speech (LIMMITS’23) challenge is organized as part of the ICASSP 2023 signal processing grand challenge. LIMMITS’23 aims at the development of a lightweight, multi-speaker, multi-lingual Text to Speech (TTS) model using datasets in Marathi, Hindi, and Telugu. The challenge encourages the advancement of TTS in Indian Languages as well as the development of techniques involved in TTS data selection and model compression. The 3 tracks of LIMMITS’23 have provided an opportunity for various researchers and practitioners around the world to explore the state of the art in TTS research.

关键词： Signal processing Data models Acoustics Speech processing

ROBUST KNOWLEDGE DISTILLATION FROM RNN-T MODELS WITH NOISY TRAINING LABELS USING FULL-SUM LOSS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zeineldeen, Mohammad Audhkhasi, Kartik Baskar, Murali Karthick Ramabhadran, Bhuvana Human Language Technology and Pattern Recognition Computer Science Department Rwth Aachen University Aachen52074 Germany Google Llc New York United States

This work studies knowledge distillation (KD) and addresses its constraints for recurrent neural network transducer (RNNT) models. In hard distillation, a teacher model transcribes large amounts of unlabelled speech to train a student model. Soft distillation is another popular KD method that distills the output logits of the teacher model. Due to the nature of RNN-T alignments, applying soft distillation between RNNT architectures having different posterior distributions is challenging. In addition, bad teachers having high word-error-rate (WER) reduce the efficacy of KD. We investigate how to effectively distill knowledge from variable quality ASR teachers, which has not been studied before to the best of our knowledge. We show that a sequence-level KD, full-sum distillation, outperforms other distillation methods for RNN-T models, especially for bad teachers. We also propose a variant of full-sum distillation that distills the sequence discriminative knowledge of the teacher leading to further improvement in WER. We conduct experiments on public datasets namely SpeechStew and LibriSpeech, and on in-house production data. Copyright © 2023, The Authors. All rights reserved.

关键词： Recurrent neural networks