检索结果-内蒙古大学图书馆

A study on the uncertainty of convolutional layers in deep neural networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Shen, Haojing Chen, Sihong Wang, Ran Big Data Institute College of Computer Science and Software Engineering Guangdong Key Lab. of Intelligent Information Processing Shenzhen University ShenzhenGuangdong518060 China College of Mathematics and Statistics Shenzhen University Shenzhen518060 China Shenzhen Key Laboratory of Advanced Machine Learning and Applications Shenzhen University Shenzhen518060 China

This paper shows a Min-Max property existing in the connection weights of the convolutional layers in a neural network structure, i.e., the LeNet. Specifically, the Min-Max property means that, during the back propagation-based training for LeNet, the weights of the convolutional layers will become far away from their centers of intervals, i.e., decreasing to their minimum or increasing to their maximum. From the perspective of uncertainty, we demonstrate that the Min-Max property corresponds to minimizing the fuzziness of the model parameters through a simplified formulation of convolution. It is experimentally confirmed that the model with the Min-Max property has a stronger adversarial robustness, thus this property can be incorporated into the design of loss function. This paper points out a changing tendency of uncertainty in the convolutional layers of LeNet structure, and gives some insights to the interpretability of convolution. © 2020, CC-BY.

关键词： Convolution

Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is availab.e at this url. © 2024, CC BY.

关键词： Semantics

Syntax-enhanced Pre-trained model

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Xu, Zenan Guo, Daya Tang, Duyu Su, Qinliang Shou, Linjun Gong, Ming Zhong, Wanjun Quan, Xiaojun Jiang, Daxin Duan, Nan School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Microsoft Research Asia Beijing China Microsoft Search Technology Center Asia Beijing China Guangdong Key Laboratory Big Data Analysis and Processing Guangzhou China Key Lab. of Machine Intelligence and Advanced Computing Ministry of Education China

We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they suffer from discrepancy between the two stages. Such a problem would lead to the necessity of having human-annotated syntactic information, which limits the application of existing methods to broader scenarios. To address this, we present a model that utilizes the syntax of text in both pre-training and fine-tuning stages. Our model is based on Transformer with a syntax-aware attention layer that considers the dependency tree of the text. We further introduce a new pre-training task of predicting the syntactic distance among tokens in the dependency tree. We evaluate the model on three downstream tasks, including relation classification, entity typing, and question answering. Results show that our model achieves state-of-the-art performance on six public benchmark datasets. We have two major findings. First, we demonstrate that infusing automatically produced syntax of text improves pre-trained models. Second, global syntactic distances among tokens bring larger performance gains compared to local head relations between contiguous tokens. Copyright © 2020, The Authors. All rights reserved.

关键词： Syntactics

Research on 3D geometric modeling of urban buildings based on airborne lidar point cloud and image 4

学校读者我要写书评

暂无评论

Research on 3D geometric modeling of urban buildings based o...

4th International Conference on Geology, Mapping, and Remote Sensing, ICGMRS 2023

作者： Guo, Tianwei Dong, Kunfeng College of Mapping and Information Engineering West Yunnan University of Applied Sciences Yunnan Province Dali China Key Lab. of Mt. Real Scene Point Cloud Data Proc. and Applic. for Universities in Yunnan Province West Yunnan University of Applied Sciences Yunnan Province Dali China Multi Src. Data Fusion Real Scene 3D Constr. Research Scientific and Technological Innovation Team West Yunnan University of Applied Sciences Yunnan Province Dali China Yunnan Construction Investment First Investigation and Design Co. Yunnan Province Kunming China

ISBN: (纸本)9781510672741

Buildings are the most important elements in cities. Building urban building models is of great significance for the establishment of digital cities. The level of its modeling technology restricts the development of urban 3D modeling technology. In order to improve the accuracy of 3D geometric modeling of urban buildings and truly reflect the spatial layout of urban buildings, the 3D geometric modeling of urban buildings based on airborne LiDAR point cloud and image is studied. The airborne LiDAR and tilt photogrammetry technology are analyzed, and the airborne LiDAR point cloud data is filtered by using the two-level grid to eliminate the gross error, and the airborne LiDAR point cloud data is filtered by using the morphological filtering algorithm. Through the principle of spatial location, the spatial geographic location information of the oblique image is restored. Extract the step line feature of the urban building point cloud and the roof patch of the urban building, accurately register and locate the urban building model based on the inclined image, and combine the urban building texture map to realize the three-dimensional geometric modeling of the urban building. The experimental results show that the proposed method has good 3D geometric modeling effect of urban buildings, can truly reflect the spatial layout of urban buildings, and effectively improve the accuracy and efficiency of 3D geometric modeling of urban buildings. © 2024 COPYRIGHT SPIE. Downloading of the abstract is permitted for personal use only.

关键词： Buildings

Incorporating Hidden Layer representation into Adversarial Attacks and Defences

学校读者我要写书评

暂无评论

arXiv 2020年

In this paper, we propose a defence strategy to improves adversarial robustness incorporating hidden layer representation. The key of this defence strategy aims to compress or filter input’s information including adversarial perturbation. And this defence strategy can be regarded as an activation function which can be applied any kind of neural networks. We also prove theoretically the effectiveness of this defense strategy under certain conditions. Besides, incorporating hidden layer representation we propose three types of adversarial attacks to generate three types of adversarial examples, respectively. The experiments show that our defence method can significantly improve the adversarial robustness of deep neural networks which achieves the state-of-the-art performance even though we do not adopt adversarial training. © 2020, CC BY.

关键词： Deep neural networks

Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Yang, Zhiyong Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China School of Cyber Science and Tech. Sun Yat-sen University Shenzhen Campus China

Diffusion models are powerful generative models, and this capability can also be applied to discrimination. The inner activations of a pre-trained diffusion model can serve as features for discriminative tasks, namely, diffusion feature. We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. To be specific, there are content differences between features and the input image, such as the exact shape of a certain object. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. Further empirical study also indicates that its negative impact is not negligible even when content shift is not visually perceivable. Hence, we propose to suppress content shift to enhance the overall quality of diffusion features. Specifically, content shift is related to the information drift during the process of recovering an image from the noisy input, pointing out the possibility of turning off-the-shelf generation techniques into tools for content shift suppression. We further propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology. Despite the simplicity, the proposed approach has achieved superior results on various tasks and datasets, validating its potential as a generic booster for diffusion features. Our code is availab.e at this url. © 2024, CC BY.

关键词：

Biogeography-based optimization for cluster analysis

学校读者我要写书评

暂无评论

Biogeography-based optimization for cluster analysis

International Conference on Computer, Communication and Computational Sciences, ICCCCS 2016

作者： Wu, Xueyan Wang, Hainan Chen, Zhimin Lu, Zhihai Phillips, Preetha Wang, Shuihua Zhang, Yudong School of Computer Science and Technology Nanjing Normal University NanjingJiangsu210023 China Key Laboratory of Statistical Information Technology & Data Mining State Statistics Bureau ChengduSichuan610225 China School of Computer and Information Engineering Henan Normal University XinxiangHenan453000 China School of Electronic Information Shanghai Dianji University Shanghai200240 China Key Laboratory of Symbolic Computation & Knowledge Engineering of Ministry of Education Jilin University ChangchunJilin130012 China Guangxi Key Laboratory of Manufacturing System & Advanced Manufacturing Technology GuilinGuangxi541004 China School of Natural Sciences and Mathematics Shepherd University ShepherdstownWV25443 United States West Virginia School of Osteopathic Medicine 400 N Lee St LewisburgWV24901 United States Department of Electrical Engineering The City College of New York CUNY New YorkNY10031 United States State Key Lab of CAD & CG Zhejiang University HangzhouZhejiang310027 China Manchester Metropolitan University ManchesterM156BH United Kingdom

ISBN: (纸本)9789811037696

With the aim of resolving the issue of cluster analysis more precisely and validly, a new approach was proposed based on biogeography-based optimization (abbreviated as BBO) algorithm. (Method) First, we reformulated the problem with an optimization model based on the variance ratio criterion (VARAC). Then, BBO was presented to search the optimal solution of the VARAC. There are 400 data of four groups in the experimental dataset, which have the degrees of overlapping of three distinct scales. The first one is nonoverlapping, the second one is partial overlapping, and the last is severely overlapping. BBO algorithm was compared with three different state-of-the-art approaches. We ran every algorithm 20 times. In this experiment, our results demonstrate the maximum VARAC values that can be found by BBO. The conclusion is that BBO is predominant which is extremely quick for the issue of clustering analysis. © Springer Nature Singapore Pte Ltd. 2017.

关键词： Cluster analysis

Accurate Latent Factor Analysis via Particle Swarm Optimizers

学校读者我要写书评

暂无评论

Accurate Latent Factor Analysis via Particle Swarm Optimizer...

IEEE International Conference on Systems, Man and Cybernetics

作者： Jia Chen Xin Luo MengChu Zhou School of Cyber Science and Technology Beihang University Beijing China Chongqing Key Lab. of Big Data and Intelligent Computing and the Chongqing Engineering Research Center of Big Data Application for Smart Cities Chongqing Institute of Green and Intelligent Technology Chongqing School University of Chinese Academy of Sciences Chongqing China New Jersey Institute of Technology Newark NJ USA

ISBN: (纸本)9781665442084

A stochastic-gradient-descent-based Latent Factor Analysis (LFA) model is highly efficient in representative learning of a High-Dimensional and Sparse (HiDS) matrix. Its learning rate adaptation is vital in ensuring its efficiency. Such adaptation can be realized with an evolutionary computing algorithm. However, a resultant model tends to suffer from two issues: a) the pre-mature convergence of the swarm of learning rates as caused by an adopted evolution algorithm, and b) the pre-mature convergence of the LFA model as caused jointly by evolution-based learning rate adaptation and an optimization algorithm. This paper focuses on the methods to address such issues. A Hierarchical Particle-swarm-optimization-incorporated Latent factor analysis (HPL) model with a two-layered structure is proposed, where the first layer pre-trains desired latent factors with a position-transitional particle-swarm-optimization-based LFA model, and the second layer performs latent factor refining with a newly-proposed mini-batch particle swarm optimizer. With such design, an HPL model can well handle the pre-mature convergence, which is supported by the positive experimental results achieved on HiDS matrices from industrial applications.

关键词： Adaptation models Analytical models Computational modeling Refining Predictive models Sparse matrices Particle swarm optimization

学校读者我要写书评

暂无评论

Preface

计算机科学技术学报（英文版） 2007年第2期22卷插1-插2页

作者： Shan Wang Jian-Zhong Li Scholl of Information Key Lab Data Engineering and Knowledge EngineeringRenmin University of China Department of Computer Science and Technology Harbin Institute of Technology

Recent advances in database related applications propose many new challenges and have inspired database researchers and practitioners to further make their efforts on new database technologies.

关键词：