检索结果-内蒙古大学图书馆

InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Renzhi Li, Jing Li, Piji College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China Department of Computing The Hong Kong Polytechnic University China

Diffusion models have garnered considerable interest in the field of text generation. Several studies have explored text diffusion models with different structures and applied them to various tasks, including named entity recognition and summarization. However, there exists a notable disparity between the "easy-first" text generation process of current diffusion models and the "keyword-first" natural text generation process of humans, which has received limited attention. To bridge this gap, we propose InfoDiffusion, a non-autoregressive text diffusion model. Our approach introduces a "keyinfo-first" generation strategy and incorporates a noise schedule based on the amount of text information. In addition, InfoDiffusion combines self-conditioning with a newly proposed partially noising model structure. Experimental results show that InfoDiffusion outperforms the baseline model in terms of generation quality and diversity, as well as exhibiting higher sampling efficiency.1 Copyright © 2023, The Authors. All rights reserved.

关键词： Character recognition

A Tailored Physics-informed Neural Network Method for Solving Singularly Perturbed Differential Equations 22

学校读者我要写书评

暂无评论

A Tailored Physics-informed Neural Network Method for Solvin...

Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial intelligence

作者： Yiwen Pang Ye Li Sheng-Jun Huang College of Computer Science and Technology/Artificial Intelligence Nanjing University of Aeronautics and Astronautics China College of Computer Science and Technology/Artificial Intelligence Nanjing University of Aeronautics and Astronautics China and MIIT Key Laboratory of Pattern Analysis and Machine Intelligence China

ISBN: (纸本)9781450398336

Physics-informed neural networks (PINNs) have recently been demonstrated to be effective for the numerical solution of differential equations, with the advantage of small real labelled data needed. However, the performance of PINN greatly depends on the differential equation. The solution of singularly perturbed differential equations (SPDEs) usually contains a boundary layer, which makes it difficult for PINN to approximate the solution of SPDEs. In this paper, we analyse the reasons for the failure of PINN in solving SPDE and provide a feasible solution by adding prior knowledge of the boundary layer to the neural network. The new method is called the tailored physics-informed neural network (TPINN) since the network is tailored to some particular properties of the problem. Numerical experiments show that our method can effectively improve both the training speed and accuracy of neural networks.

关键词： physics-informed neural network prior knowledge boundary layer singularly perturbed differential equation

Improving model robustness by adaptively correcting perturbation levels with active queries

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ning, Kun-Peng Tao, Lue Chen, Songcan Huang, Sheng-Jun College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing211106 China

In addition to high accuracy, robustness is becoming increasingly important for machine learning models in various applications. Recently, much research has been devoted to improving the model robustness by training with noise perturbations. Most existing studies assume a fixed perturbation level for all training examples, which however hardly holds in real tasks. In fact, excessive perturbations may destroy the discriminative content of an example, while deficient perturbations may fail to provide helpful information for improving the robustness. Motivated by this observation, we propose to adaptively adjust the perturbation levels for each example in the training process. Specifically, a novel active learning framework is proposed to allow the model to interactively query the correct perturbation level from human experts. By designing a cost-effective sampling strategy along with a new query type, the robustness can be significantly improved with a few queries. Both theoretical analysis and experimental studies validate the effectiveness of the proposed approach. © 2021, CC BY.

关键词： Cost effectiveness

MUS-CDB: Mixed Uncertainty Sampling with Class Distribution Balancing for Active Annotation in Aerial Object Detection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liang, Dong Zhang, Jing-Wei Tang, Ying-Peng Huang, Sheng-Jun The College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing211106 China

Recent aerial object detection models rely on a large amount of labeled training data, which requires unaffordable manual labeling costs in large aerial scenes with dense objects. Active learning effectively reduces the data labeling cost by selectively querying the informative and representative unlabelled samples. However, existing active learning methods are mainly with class-balanced settings and image-based querying for generic object detection tasks, which are less applicable to aerial object detection scenarios due to the long-tailed class distribution and dense small objects in aerial scenes. In this paper, we propose a novel active learning method for cost-effective aerial object detection. Specifically, both object-level and image-level informativeness are considered in the object selection to refrain from redundant and myopic querying. Besides, an easy-to-use class-balancing criterion is incorporated to favor the minority objects to alleviate the long-tailed class distribution problem in model training. We further devise a training loss to mine the latent knowledge in the unlabeled image regions. Extensive experiments are conducted on the DOTA-v1.0 and DOTA-v2.0 benchmarks to validate the effectiveness of the proposed method. For the ReDet, KLD, and SASM detectors on the DOTAv2.0 dataset, the results show that our proposed MUS-CDB method can save nearly 75% of the labeling cost while achieving comparable performance to other active learning methods in terms of mAP. Code is publicly online. Copyright © 2022, The Authors. All rights reserved.

关键词： Object detection

Truly proximal policy optimization

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wang, Yuhui He, Hao Wen, Chao Tan, Xiaoyang College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing210016 China

Proximal policy optimization (PPO) is one of the most successful deep reinforcement-learning methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, its optimization behavior is still far from being fully understood. In this paper, we show that PPO could neither strictly restrict the likelihood ratio as it attempts to do nor enforce a well-defined trust region constraint, which means that it may still suffer from the risk of performance instability. To address this issue, we present an enhanced PPO method, named Truly PPO. Two critical improvements are made in our method: 1) it adopts a new clipping function to support a rollback behavior to restrict the difference between the new policy and the old one;2) the triggering condition for clipping is replaced with a trust region-based one, such that optimizing the resulted surrogate objective function provides guaranteed monotonic improvement of the ultimate policy performance. It seems, by adhering more truly to making the algorithm proximal - confining the policy within the trust region, the new algorithm improves the original PPO on both sample efficiency and performance. Copyright © 2019, The Authors. All rights reserved.

关键词： Deep learning

Characteristic AI Agents via Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Xi Dai, Hongliang Gao, Shen Li, Piji College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China School of Computer Science and Technology Shandong University China

The advancement of Large Language Models (LLMs) has led to significant enhancements in the performance of chatbot systems. Many researchers have dedicated their efforts to the development of bringing characteristics to chatbots. While there have been commercial products for developing role-driven chatbots using LLMs, it is worth noting that academic research in this area remains relatively scarce. Our research focuses on investigating the performance of LLMs in constructing Characteristic AI Agents by simulating real-life individuals across different settings. Current investigations have primarily focused on act on roles with simple profiles. In response to this research gap, we create a benchmark for the characteristic AI agents task, including dataset, techniques, and evaluation metrics. A dataset called "Character100" is built for this benchmark, comprising the most-visited people on Wikipedia for language models to role-play. With the constructed dataset, we conduct comprehensive assessment of LLMs across various settings. In addition, we devise a set of automatic metrics for quantitative performance evaluation. The experimental results underscore the potential directions for further improvement in the capabilities of LLMs in constructing characteristic AI agents. The benchmark is available at https://***/nuaa-nlp/Character100. © 2024, CC0.

关键词： Computational linguistics

With false friends like these, who can notice mistakes?

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Tao, Lue Feng, Lei Yi, Jinfeng Chen, Songcan College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science Chongqing University China JD AI Research

Adversarial examples crafted by an explicit adversary have attracted significant attention in machine learning. However, the security risk posed by a potential false friend has been largely overlooked. In this paper, we unveil the threat of hypocritical examples-inputs that are originally misclassified yet perturbed by a false friend to force correct predictions. While such perturbed examples seem harmless, we point out for the first time that they could be maliciously used to conceal the mistakes of a substandard (i.e., not as good as required) model during an evaluation. Once a deployer trusts the hypocritical performance and applies the "well-performed" model in realworld applications, unexpected failures may happen even in benign environments. More seriously, this security risk seems to be pervasive: we find that many types of substandard models are vulnerable to hypocritical examples across multiple datasets. Furthermore, we provide the first attempt to characterize the threat with a metric called hypocritical risk and try to circumvent it via several countermeasures. Results demonstrate the effectiveness of the countermeasures, while the risk remains non-negligible even after adaptive robust training. Copyright © 2020, The Authors. All rights reserved.

关键词： machine learning

Guidance Not Obstruction: A Conjugate Consistent Enhanced Strategy for Domain Generalization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cao, Meng Chen, Songcan The MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing University of Aeronautics and Astronautics Nanjing210016 China The College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing210016 China

Domain generalization addresses domain shift in real-world applications. Most approaches adopt a domain angle, seeking invariant representation across domains by aligning their marginal distributions, irrespective of individual classes, naturally leading to insufficient exploration of discriminative information. Switching to a class angle, we find that multiple domain-related peaks or clusters within the same individual classes must emerge due to distribution shift. In other words, marginal alignment does not guarantee conditional alignment, leading to suboptimal generalization. Therefore, we argue that acquiring discriminative generalization between classes within domains is crucial. In contrast to seeking distribution alignment, we endeavor to safeguard domain-related between-class discrimination. To this end, we devise a novel Conjugate Consistent Enhanced Module, namely Con2EM, based on a distribution over domains, i.e., a meta-distribution. Specifically, we employ a novel distribution-level Universum strategy to generate supplementary diverse domain-related class-conditional distributions, thereby enhancing generalization. This allows us to resample from these generated distributions to provide feedback to the primordial instance-level classifier, further improving its adaptability to the target-agnostic. To ensure generation accuracy, we establish an additional distribution-level classifier to regularize these conditional distributions. Extensive experiments have been conducted to demonstrate its effectiveness and low computational cost compared to SOTAs. Copyright © 2024, The Authors. All rights reserved.

关键词：

Deep robust multilevel semantic cross-modal hashing

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Song, Ge Zhao, Jun Tan, Xiaoyang College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Miit Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization Nanyang Technological University

Hashing based cross-modal retrieval has rec made significant progress. But straightfor embedding data from different modalities in joint Hamming space will inevitably produce codes due to the intrinsic modality discrep and noises. We present a novel Robust M level Semantic Hashing (RMSH) for more a rate cross-modal retrieval. It seeks to pres fine-grained similarity among data with rich mantics, while explicitly require distances bet dissimilar points to be larger than a specific v for strong robustness. For this, we give an fective bound of this value based on the information coding-theoretic analysis, and the above goals are embodied into a margin-adaptive triplet loss. Furthermore, we introduce pseudo-codes via fusing multiple hash codes to explore seldom-seen semantics, alleviating the sparsity problem of similarity information. Experiments on three benchmarks show the validity of the derived bounds, and our method achieves state-of-the-art performance. Copyright © 2020, The Authors. All rights reserved.

关键词： Semantics