检索结果-内蒙古大学图书馆

Towards Feature Distribution Alignment and Diversity Enhancement for data-Free Quantization

学校读者我要写书评

暂无评论

Towards Feature Distribution Alignment and Diversity Enhance...

IEEE International Conference on data Mining (ICDM)

作者： Yangcheng Gao Zhao Zhang Richang Hong Haijun Zhang Jicong Fan Shuicheng Yan School of Computer Science and Information Engineering Hefei University of Technology Hefei China Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education) & Intelligent Interconnected Systems Laboratory of Anhui Province Hefei University of Technology Hefei China Department of Computer Science Harbin Institute of Technology (Shenzhen) Xili University Town Shenzhen China School of Data Science The Chinese University of Hong Kong Shenzhen China Shenzhen Research Institute of Big Data Shenzhen China National University of Singapore Singapore

To obtain lower inference latency and less memory footprint of deep neural networks, model quantization has been widely employed in deep model deployment, by converting the floating points to low-precision integers. However, previous methods (such as quantization aware training and post training quantization) require original data for the fine-tuning or calibration of quantized model, which makes them inapplicable to the cases that original data are not accessed due to privacy or security. This gives birth to the data-free quantization method with synthetic data generation. While current data-free quantization methods still suffer from severe performance degradation when quantizing a model into lower bit, caused by the low inter-class separability of semantic features. To this end, we propose a new and effective data-free quantization method termed ClusterQ, which utilizes the feature distribution alignment for synthetic data generation. To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics to imitate the distribution of real data, so that the performance degradation is alleviated. Moreover, we incorporate the diversity enhancement to solve class-wise mode collapse. We also employ the exponential moving average to update the centroid of each cluster for further feature distribution improvement. Extensive experiments based on different deep models (e.g., ResNet-18 and MobileNet-V2) over the ImageNet dataset demonstrate that our proposed ClusterQ model obtains state-of-the-art performance.

关键词： Degradation Training Quantization (signal) Semantics Neural networks Memory management data models

Joint-Motion Mutual Learning for Pose Estimation in Videos

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Sifan Chen, Haipeng Yin, Yifang Hu, Sihao Feng, Runyang Jiao, Yingying Yang, Ziqi Liu, Zhenguang College of Computer Science and Technology Jilin University Changchun China A*STAR Singapore Georgia Institute of Technology Atlanta United States School of Artificial Intelligence Jilin University Changchun China The State Key Laboratory of Blockchain and Data Security Zhejiang University Hangzhou China Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University Changchun China Institute of Blockchain and Data Security Hangzhou China

Human pose estimation in videos has long been a compelling yet challenging task within the realm of computer vision. Nevertheless, this task remains difficult because of the complex video scenes, such as video defocus and self-occlusion. Recent methods strive to integrate multi-frame visual features generated by a backbone network for pose estimation. However, they often ignore the useful joint information encoded in the initial heatmap, which is a byproduct of the backbone generation. Comparatively, methods that attempt to refine the initial heatmap fail to consider any spatiotemporal motion features. As a result, the performance of existing methods for pose estimation falls short due to the lack of ability to leverage both local joint (heatmap) information and global motion (feature) dynamics. To address this problem, we propose a novel joint-motion mutual learning framework for pose estimation, which effectively concentrates on both local joint dependency and global pixel-level motion dynamics. Specifically, we introduce a context-aware joint learner that adaptively leverages initial heatmaps and motion flow to retrieve robust local joint feature. Given that local joint feature and global motion flow are complementary, we further propose a progressive joint-motion mutual learning that synergistically exchanges information and interactively learns between joint feature and motion flow to improve the capability of the model. More importantly, to capture more diverse joint and motion cues, we theoretically analyze and propose an information orthogonality objective to avoid learning redundant information from multi-cues. Empirical experiments show our method outperforms prior arts on three challenging benchmarks. Copyright © 2024, The Authors. All rights reserved.

关键词： Motion estimation

Towards certifying ∞ robustness using neural networks with ∞-dist neurons

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Bohang Cai, Tianle Lu, Zhou He, Di Wang, Liwei Key Laboratory of Machine Perception MOE School of EECS Peking University Department of Electrical and Computer Engineering Princeton University Zhongguancun Haihua Institute for Frontier Information Technology Department of Computer Science Princeton University Microsoft Research Center for Data Science Peking University

It is well-known that standard neural networks, even with a high classification accuracy, are vulnerable to small ∞-norm bounded adversarial perturbations. Although many attempts have been made, most previous works either can only provide empirical verification of the defense to a particular attack method, or can only develop a certified guarantee of the model robustness in limited scenarios. In this paper, we seek for a new approach to develop a theoretically principled neural network that inherently resists ∞ perturbations. In particular, we design a novel neuron that uses ∞-distance as its basic operation (which we call ∞-dist neuron), and show that any neural network constructed with ∞-dist neurons (called ∞-dist net) is naturally a 1-Lipschitz function with respect to ∞-norm. This directly provides a rigorous guarantee of the certified robustness based on the margin of prediction outputs. We then prove that such networks have enough expressive power to approximate any 1-Lipschitz function with robust generalization guarantee. We further provide a holistic training strategy that can greatly alleviate optimization difficulties. Experimental results show that using ∞-dist nets as basic building blocks, we consistently achieve state-of-the-art performance on commonly used datasets: 93.09% certified accuracy on MNIST (∊ = 0.3), 35.42% on CIFAR-10 (∊ = 8/255) and 16.31% on TinyImageNet (∊ = 1/255). Copyright © 2021, The Authors. All rights reserved.

关键词： Neurons

Finding the proper node ranking method for complex networks

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yu, Senbin Gao, Liang Wang, Yi-Fan Institute of Transportation Systems Science and Engineering MOE Key Laboratory of Urban Transportation System Theory and Technology State Key Laboratory of Rail Traffic Control and Safety Center of Cooperative Innovation for Beijing Metropolitan Transportation Beijing Jiaotong University Beijing100044 China Yinchuan Municipal Bureau of Big Data Management and Service Yinchuan750011 China

Ranking node importance is crucial in understanding network structure and function on complex networks. Degree, h-index and coreness are widely used, but which one is more proper to a network associated with a dynamical process, e.g. SIR spreading process, is still unclear. To fill this gap, a, which is extracted from the fitting 1 function (f(x) = 1 − e2a(x−b)+1) of the average number of nodes in each radius of the neighborhood of a node, is proposed. Experiment results which are carried out on twenty real-world networks show that a can classify which of the three measures (degree, h-index and coreness) is more proper to a network in ranking node importance. We also find that [b/3] is a good indicator for forecasting the optimal radius of the neighborhood of a node in ranking node importance for a given network. To the best of our knowledge, it is the first solution of this interesting and open issue. Furthermore, by extending the range of neighborhood where we construct an operator ℋ on of a node, we propose a new method to quantify the importance of a node. The ranking accuracies of most networks can be improved when the radius is increased from 0 to its forecasting optimal radius and the improvement, for the best case, reaches up to 111%. The performances will reduce on half of the networks studied in this paper if we roughly extend the radius of the neighborhood. Our work deepens the understanding of how to find out the proper node ranking method for complex networks. The proposed methods bridge the gaps among network structure and node importance, and may have potential applications in controlling the outbreak of disease, designing of optimal information spreading *** Codes Game theory, economics, social and behavioral sciences Copyright © 2018, The Authors. All rights reserved.

关键词： Complex networks

Local articulation points in complex networks

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yu, Senbin Gao, Liang Song, Rongqiu Institute of Transportation Systems Science and Engineering MOE Key Laboratory of Urban Transportation System Theory and Technology State Key Laboratory of Rail Traffic Control and Safety Center of Cooperative Innovation for Beijing Metropolitan Transportation Beijing Jiaotong University Beijing100044 China Yinchuan Municipal Bureau of Big Data Management and Service Yinchuan750011 China

An articulation point (AP) is any node whose removal increases the number of connected components of a graph. There is no doubt that this kind of node which occupies a non-ignorable fraction of real-world networks plays a key role in ensuring the connectivity. However, we should not thus neglect the impacts of non-APs nodes. In this paper, we define a local AP (LAP) whose removal will increase the number of connected components within its r-step neighborhood. Through investigating the fraction of LAPs in forty-five real networks, we find a critical proportion Scr, which is equal to 0.5 (S = r/D, D is the diameter of a network), and this result can also be turned out in ER networks. In addition, we present a unique advantage of LAPs in dismantling networks under the process of targeted attack, compared with APs, which provide another way of thinking to improve the calculation efficiency of APs and design better targeted attack strategy of network *** Codes Game theory, economics, social and behavioral sciences Copyright © 2018, The Authors. All rights reserved.

关键词： Complex networks

Document-level Relation Extraction with Relation Correlations

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Han, Ridong Peng, Tao Wang, Benyou Liu, Lu Wan, Xiang College of Computer Science and Technology Jilin University China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education China College of Software Jilin University China Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China School of Data Science The Chinese University of Hong Kong Shenzhen China

Document-level relation extraction faces two overlooked challenges: long-tail problem and multi-label problem. Previous work focuses mainly on obtaining better contextual representations for entity pairs, hardly address the above challenges. In this paper, we analyze the co-occurrence correlation of relations, and introduce it into DocRE task for the first time. We argue that the correlations can not only transfer knowledge between data-rich relations and data-scarce ones to assist in the training of tailed relations, but also reflect semantic distance guiding the classifier to identify semantically close relations for multi-label entity pairs. Specifically, we use relation embedding as a medium, and propose two co-occurrence prediction sub-tasks from both coarse- and fine-grained perspectives to capture relation correlations. Finally, the learned correlation-aware embeddings are used to guide the extraction of relational facts. Substantial experiments on two popular DocRE datasets are conducted, and our method achieves superior results compared to baselines. Insightful analysis also demonstrates the potential of relation correlations to address the above challenges. Copyright © 2022, The Authors. All rights reserved.

关键词： Extraction

(Locally) Differentially Private Combinatorial Semi-Bandits

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Chen, Xiaoyu Zheng, Kai Zhou, Zixin Yang, Yunchang Chen, Wei Wang, Liwei Key Laboratory of Machine Perception MOE School of EECS Peking University Work done while interned at Microsoft Research Asia School of Electronics Engineering and Computer Science Peking University Center for Data Science Peking University Microsoft Research Asia Beijing China

In this paper, we study Combinatorial Semi-Bandits (CSB) that is an extension of classic Multi-Armed Bandits (MAB) under Differential Privacy (DP) and stronger Local Differential Privacy (LDP) setting. Since the server receives more information from users in CSB, it usually causes additional dependence on the dimension of data, which is a notorious side-effect for privacy preserving learning. However for CSB under two common smoothness assumptions (Kveton et al., 2015;Chen et al., 2016), we show it is possible to remove this side-effect. In detail, for B∞-bounded smooth CSB under either Ε-LDP or Ε-DP, we prove the optimal regret bound is Θ(mB∆∞2Ε2ln T ) or Θ( mB∆∞2Εln T ) respectively, where T is time period, ∆ is the gap of rewards and m is the number of base arms, by proposing novel algorithms and matching lower bounds. For B1-bounded smooth CSB under Ε-DP, we also prove the optimal regret bound is Θ( mKB∆1 2Εln T ) with both upper bound and lower bound, where K is the maximum number of feedback in each round. All above results nearly match corresponding non-private optimal rates, which imply there is no additional price for (locally) differentially private CSB in above common settings. Copyright © 2020, The Authors. All rights reserved.

关键词： Machine learning

Biogeography-based optimization for cluster analysis

学校读者我要写书评

暂无评论

Biogeography-based optimization for cluster analysis

International Conference on Computer, Communication and Computational Sciences, ICCCCS 2016

作者： Wu, Xueyan Wang, Hainan Chen, Zhimin Lu, Zhihai Phillips, Preetha Wang, Shuihua Zhang, Yudong School of Computer Science and Technology Nanjing Normal University NanjingJiangsu210023 China Key Laboratory of Statistical Information Technology & Data Mining State Statistics Bureau ChengduSichuan610225 China School of Computer and Information Engineering Henan Normal University XinxiangHenan453000 China School of Electronic Information Shanghai Dianji University Shanghai200240 China Key Laboratory of Symbolic Computation & Knowledge Engineering of Ministry of Education Jilin University ChangchunJilin130012 China Guangxi Key Laboratory of Manufacturing System & Advanced Manufacturing Technology GuilinGuangxi541004 China School of Natural Sciences and Mathematics Shepherd University ShepherdstownWV25443 United States West Virginia School of Osteopathic Medicine 400 N Lee St LewisburgWV24901 United States Department of Electrical Engineering The City College of New York CUNY New YorkNY10031 United States State Key Lab of CAD & CG Zhejiang University HangzhouZhejiang310027 China Manchester Metropolitan University ManchesterM156BH United Kingdom

ISBN: (纸本)9789811037696

With the aim of resolving the issue of cluster analysis more precisely and validly, a new approach was proposed based on biogeography-based optimization (abbreviated as BBO) algorithm. (Method) First, we reformulated the problem with an optimization model based on the variance ratio criterion (VARAC). Then, BBO was presented to search the optimal solution of the VARAC. There are 400 data of four groups in the experimental dataset, which have the degrees of overlapping of three distinct scales. The first one is nonoverlapping, the second one is partial overlapping, and the last is severely overlapping. BBO algorithm was compared with three different state-of-the-art approaches. We ran every algorithm 20 times. In this experiment, our results demonstrate the maximum VARAC values that can be found by BBO. The conclusion is that BBO is predominant which is extremely quick for the issue of clustering analysis. © Springer Nature Singapore Pte Ltd. 2017.

关键词： Cluster analysis

JDsearch: A Personalized Product Search dataset with Real Queries and Full Interactions

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Jiongnan Dou, Zhicheng Tang, Guoyu Xu, Sulong Gaoling School of Artificial Intelligence Renmin University of China Beijing China Engineering Research Center of Next-Generation Intelligent Search and Recommendation MOE Beijing China *** Inc. Beijing China Beijing Key Laboratory of Big Data Management and Analysis Methods China

Recently, personalized product search attracts great attention and many models have been proposed. To evaluate the effectiveness of these models, previous studies mainly utilize the simulated Amazon recommendation dataset, which contains automatically generated queries and excludes cold users and tail products. We argue that evaluating with such a dataset may yield unreliable results and conclusions, and deviate from real user satisfaction. To overcome these problems, in this paper, we release a personalized product search dataset comprised of real user queries and diverse user-product interaction types (clicking, adding to cart, following, and purchasing) collected from ***, a popular Chinese online shopping platform. More specifically, we sample about 170,000 active users on a specific date, then record all their interacted products and issued queries in one year, without removing any tail users and products. This finally results in roughly 12,000,000 products, 9,400,000 real searches, and 26,000,000 user-product interactions. We study the characteristics of this dataset from various perspectives and evaluate representative personalization models to verify its feasibility. The dataset can be publicly accessed at Github: https://***/rucliujn/JDsearch. © 2023, CC BY-NC-SA.

关键词：