检索结果-内蒙古大学图书馆

Dense residual network: Enhancing global dense feature flow for character recognition

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhang, Zhao Tang, Zemin Wang, Yang Zhang, Zheng Zhan, Choujun Zha, Zhengjun Wang, Meng School of Computer Science and Information Engineering Hefei University of Technology Hefei230009 China Key Laboratory of Knowledge Engineering with Big Data Ministry of Education Intelligent Interconnected Systems Laboratory of Anhui Province Hefei University of Technology Hefei230009 China School of Computer Science and Technology Soochow University Suzhou215006 China Shenzhen China School of Computer South China Normal University Guangzhou510631 China Deparmtment of Computer Science and Technology University of Science and Technology of China Hefei China

Deep Convolutional Neural Networks (CNNs), such as Dense Convolutional Network (DenseNet), have achieved great success for image representation learning by capturing deep hierarchical features. However, most existing network architectures of simply stacking the convolutional layers fail to enable them to fully discover local and global feature information between layers. In this paper, we mainly investigate how to enhance the local and global feature learning abilities of DenseNet by fully exploiting the hierarchical features from all convolutional layers. Technically, we propose an effective convolutional deep model termed Dense Residual Network (DRN) for the task of optical character recognition. To define DRN, we propose a refined residual dense block (r-RDB) to retain the ability of local feature fusion and local residual learning of original RDB, which can reduce the computing efforts of inner layers at the same time. After fully capturing local residual dense features, we utilize the sum operation and several r-RDBs to construct a new block termed global dense block (GDB) by imitating the construction of dense blocks to adaptively learn global dense residual features in a holistic way. Finally, we use two convolutional layers to design a down-sampling block to reduce the global feature size and extract more informative deeper features. Extensive results show that our DRN can deliver enhanced results, compared with other related deep models. Copyright © 2020, The Authors. All rights reserved.

关键词： Convolution

Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is available at this url. © 2024, CC BY.

关键词： Semantics

AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hou, Wenzheng Xu, Qianqian Yang, Zhiyong Bao, Shilong He, Yuan Huang, Qingming Key Laboratory of Intelligent Information Processing Institute of Computing Technology CAS Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China State Key Laboratory of Information Security Institute of Information Engineering CAS Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Alibaba Group Beijing China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing China Artificial Intelligence Research Center Peng Cheng Laboratory Shenzhen China

It is well-known that deep learning models are vulnerable to adversarial examples. Existing studies of adversarial training have made great progress against this challenge. As a typical trait, they often assume that the class distribution is overall balanced. However, long-tail datasets are ubiquitous in a wide spectrum of applications, where the amount of head class instances is larger than the tail classes. Under such a scenario, AUC is a much more reasonable metric than accuracy since it is insensitive toward class distribution. Motivated by this, we present an early trial to explore adversarial training methods to optimize AUC. The main challenge lies in that the positive and negative examples are tightly coupled in the objective function. As a direct result, one cannot generate adversarial examples without a full scan of the dataset. To address this issue, based on a concavity regularization scheme, we reformulate the AUC optimization problem as a saddle point problem, where the objective becomes an instance-wise function. This leads to an end-to-end training protocol. Furthermore, we provide a convergence guarantee of the proposed algorithm. Our analysis differs from the existing studies since the algorithm is asked to generate adversarial examples by calculating the gradient of a min-max problem. Finally, the extensive experimental results show the performance and robustness of our algorithm in three long-tail datasets. © 2022, CC BY.

关键词： Deep learning

Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Yang, Zhiyong Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China School of Cyber Science and Tech. Sun Yat-sen University Shenzhen Campus China

Diffusion models are powerful generative models, and this capability can also be applied to discrimination. The inner activations of a pre-trained diffusion model can serve as features for discriminative tasks, namely, diffusion feature. We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. To be specific, there are content differences between features and the input image, such as the exact shape of a certain object. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. Further empirical study also indicates that its negative impact is not negligible even when content shift is not visually perceivable. Hence, we propose to suppress content shift to enhance the overall quality of diffusion features. Specifically, content shift is related to the information drift during the process of recovering an image from the noisy input, pointing out the possibility of turning off-the-shelf generation techniques into tools for content shift suppression. We further propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology. Despite the simplicity, the proposed approach has achieved superior results on various tasks and datasets, validating its potential as a generic booster for diffusion features. Our code is available at this url. © 2024, CC BY.

关键词：

Structure-aware random fourier kernel for graphs 21

学校读者我要写书评

暂无评论

Structure-aware random fourier kernel for graphs

Proceedings of the 35th International Conference on Neural Information Processing Systems

作者： Jinyuan Fang Qiang Zhang Zaiqiao Meng Shangsong Liang School of Computer Science and Engineering Sun Yat-sen University China and Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Hangzhou Innovation Center Zhejiang University China and College of Computer Science and Technology Zhejiang University China and AZFT Knowledge Engine Lab China School of Computing Science University of Glasgow United Kingdom and Mohamed bin Zayed University of Artificial Intelligence United Arab Emirates School of Computer Science and Engineering Sun Yat-sen University China and Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China and Mohamed bin Zayed University of Artificial Intelligence United Arab Emirates

ISBN: (纸本)9781713845393

Gaussian Processes (GPs) define distributions over functions and their generalization capabilities depend heavily on the choice of kernels. In this paper, we propose a novel structure-aware random Fourier (SRF) kernel for GPs that brings several benefits when modeling graph-structured data. First, SRF kernel is defined with a spectral distribution based on the Fourier duality given by the Bochner's theorem, transforming the kernel learning problem to a distribution inference problem. Second, SRF kernel admits a random Fourier feature formulation that makes the kernel scalable for optimization. Third, SRF kernel enables to leverage geometric structures by taking subgraphs as inputs. To effectively optimize GPs with SRF kernel, we develop a variational EM algorithm, which alternates between an inference procedure (E-step) and a learning procedure (M-step). Experimental results on five real-world datasets show that our model can achieve state-of-the-art performance in two typical graph learning tasks, i.e., object classification and link prediction.

关键词：

A Survey on Fairness in Large Language Models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Yingji Du, Mengnan Song, Rui Wang, Xin Wang, Ying College of Computer Science and Technology Jilin University Changchun130012 China Department of Data Science New Jersey Institute of Technology Newark United States School of Artificial Intelligence Jilin University Changchun130012 China Key Laboratory of Symbol Computation and Knowledge Engineering Jilin University Ministry of Education Changchun130012 China

Large Language Models (LLMs) have shown powerful performance and development prospects and are widely deployed in the real world. However, LLMs can capture social biases from unprocessed training data and propagate the biases to downstream tasks. Unfair LLM systems have undesirable social impacts and potential harms. In this paper, we provide a comprehensive review of related research on fairness in LLMs. Considering the influence of parameter magnitude and training paradigm on research strategy, we divide existing fairness research into oriented to medium-sized LLMs under pre-training and fine-tuning paradigms and oriented to large-sized LLMs under prompting paradigms. First, for medium-sized LLMs, we introduce evaluation metrics and debiasing methods from the perspectives of intrinsic bias and extrinsic bias, respectively. Then, for large-sized LLMs, we introduce recent fairness research, including fairness evaluation, reasons for bias, and debiasing methods. Finally, we discuss and provide insight on the challenges and future directions for the development of fairness in LLMs. Copyright © 2023, The Authors. All rights reserved.

关键词： Computational linguistics

Uncertainty-based Model Acceleration for Cancer Classification in Whole-Slide Images

学校读者我要写书评

暂无评论

Uncertainty-based Model Acceleration for Cancer Classificati...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Zeyu Gao Anyu Mao Jialun Wu Yang Li Chunbao Wang Caixia Ding Tieliang Gong Chen Li School of Computer Science and Technology Xi’an Jiaotong University Xi’an China Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering Xi’an Jiaotong University Xi’an China Department of Pathology the First Affiliated Hospital of Xi’an Jiaotong University Xi’an China Department of Pathology Shaanxi Provincial Cancer Hospital Xi’an China

ISBN: (数字)9781665468190

ISBN: (纸本)9781665468206

Computational Pathology (CPATH) offers the possibility for highly accurate and low-cost automated pathological diagnosis. However, the high time cost of model inference is one of the main issues limiting the application of CPATH methods. Due to the large size of Whole-Slide Image (WSI), commonly used CPATH methods divided a WSI into a large number of image patches at relatively high magnification, then predicted each image patch individually, which is time-consuming. In this paper, we propose a novel Uncertainty-based Model Acceleration (UMA) method for reducing the time cost of model inference, thereby relieving the deployment burden of CPATH applications. Enlightened by the slide-viewing process of pathologists, only a few high-uncertain regions are regarded as “suspicious” regions that need to be predicted at high magnification, and most of the regions in WSI are predicted at low magnification, thereby reducing the times of image patch extraction and prediction. Meanwhile, uncertainty estimation ensures prediction accuracy at low magnification. We take two fundamental CPATH classification tasks (i.e., cancer region detection and subtyping) as examples. Extensive experiments on two large-scale renal cell carcinoma classification datasets demonstrate that our UMA can significantly reduce the time cost of model inference while maintaining competitive classification performance.

关键词： Pathology Costs Uncertainty Limiting Hospitals Biological system modeling Computational modeling

Sparse Training data-Based Hyperspectral Image Super Resolution Via ANFIS Interpolation

学校读者我要写书评

暂无评论

Sparse Training Data-Based Hyperspectral Image Super Resolut...

IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)

作者： Jing Yang Changjing Shang Lu Chen Pan Su Qiang Shen School of Automation and Software Engineering Shanxi University Taiyuan China Department of Computer Science Aberystwyth University Aberystwyth UK Insti. of Big Data Science & Industry Shanxi University Taiyuan China Department of Computer Hebei Key Laboratory of Knowledge Computing for Energy & Power Baoding North China Electric Power University China

Hyperspectral image super resolution aims to improve the spatial resolution of given hyperspectral images, which has become a highly attractive topic in the field of image processing. Existing techniques typically focus on super resolution with sufficient training data. However, restricted by data acquisition conditions, certain hyperspectral images or band images are very different to obtain, resulted in insufficient training data. In order to solve this problem, a new hyperspectral image super resolution method is proposed in this paper in an effort to conduct the super resolution task over insufficient (sparse) training data, by applying the recently introduced ANFIS (Adaptive Network-based Fuzzy Inference System) interpolation method. Particularly, the training dataset is divided into several subsets. For the subsets with sufficient training data, the relevant ANFIS models are trained using standard ANFIS learning algorithm, while for the subsets with sparse training data, the corresponding ANFIS models are interpolated through the use of ANFIS interpolation. Experimental results indicate that compared with the methods using sufficient training data, the proposed method can achieve very similar result, showing its effectiveness for situations where only sparse training data is available.

关键词：

A Tent Lévy Flying Sparrow Search Algorithm for Feature Selection: A COVID-19 Case Study

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Qinwen Gao, Yuelin Song, Yanjie School of Computer Science and Engineering North Minzu University Yinchuan750021 China School of Mathematics and Information Science North Minzu University Yinchuan750021 China Ningxia Key Laboratory of Intelligent Information and Big Data Processing Yinchuan750021 China Ningxia Province Cooperative Innovation Center of Scientific Computing and Intelligent in Formation Processing North Minzu University Yinchuan750021 China College of Systems Engineering National University of Defense Technology Changsha410073 China

The "Curse of Dimensionality" induced by the rapid development of information science, might have a negative impact when dealing with big datasets. In this paper, we propose a variant of the sparrow search algorithm (SSA), called Tent Lévy flying sparrow search algorithm (TFSSA), and use it to select the best subset of features in the packing pattern for classification purposes. SSA is a recently proposed algorithm that has not been systematically applied to feature selection problems. After verification by the CEC2020 benchmark function, TFSSA is used to select the best feature combination to maximize classification accuracy and minimize the number of selected features. The proposed TFSSA is compared with nine algorithms in the literature. Nine evaluation metrics are used to properly evaluate and compare the performance of these algorithms on twenty-one datasets from the UCI repository. Furthermore, the approach is applied to the coronavirus disease (COVID-19) dataset, yielding the best average classification accuracy and the average number of feature selections, respectively, of 93.47% and 2.1. Experimental results confirm the advantages of the proposed algorithm in improving classification accuracy and reducing the number of selected features compared to other wrapper-based algorithms. Copyright © 2022, The Authors. All rights reserved.

关键词： COVID-19