检索结果-内蒙古大学图书馆

Robust frequent directions with application in online learning

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Luo, Luo Chen, Cheng Zhang, Zhihua Li, Wu-Jun Zhang, Tong Department of Computer Science and Engineering Shanghai Jiao Tong University 800 Dongchuan Road Shanghai200240 China National Engineering Lab for Big Data Analysis and Applications School of Mathematical Sciences Peking University 5 Yiheyuan Road Beijing100871 China National Key Laboratory for Novel Software Technology Collaborative Innovation Center of Novel Software Technology and Industrialization Department of Computer Science and Technology Nanjing University 163 Xianlin Avenue Nanjing210023 China Computer Science & Mathematics Hong Kong University of Science and Technology Hong Kong

The frequent directions (FD) technique is a deterministic approach for online sketching that has many applications in machine learning. The conventional FD is a heuristic procedure that often outputs rank deficient matrices. To overcome the rank deficiency problem, we propose a new sketching strategy called robust frequent directions (RFD) by introducing a regularization term. RFD can be derived from an optimization problem. It updates the sketch matrix and the regularization term adaptively and jointly. RFD reduces the approximation error of FD without increasing the computational cost. We also apply RFD to online learning and propose an effective hyperparameter-free online Newton algorithm. We derive a regret bound for our online Newton algorithm based on RFD, which guarantees the robustness of the algorithm. The experimental studies demonstrate that the proposed method outperforms state-of-the-art second order online learning algorithms. Copyright © 2017, The Authors. All rights reserved.

关键词： Heuristic methods

iGniter: Interference-Aware GPU Resource Provisioning for Predictable DNN Inference in the Cloud

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xu, Fei Xu, Jianian Chen, Jiabin Chen, Li Shang, Ruitao Zhou, Zhi Liu, Fangming The Shanghai Key Laboratory of Multidimensional Information Processing School of Computer Science and Technology East China Normal University 3663 N. Zhongshan Road Shanghai200062 China The School of Computing and Informatics University of Louisiana at Lafayette 301 East Lewis Street LafayetteLA70504 United States The Guangdong Key Laboratory of Big Data Analysis and Processing School of Computer Science and Engineering Sun Yat-sen University 132 E. Waihuan Road Guangzhou510006 China The National Engineering Research Center for Big Data Technology and System The Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology 1037 Luoyu Road Wuhan430074 China

GPUs are essential to accelerating the latency-sensitive deep neural network (DNN) inference workloads in cloud datacenters. To fully utilize GPU resources, spatial sharing of GPUs among co-located DNN inference workloads becomes increasingly compelling. However, GPU sharing inevitably brings severe performance interference among co-located inference workloads, as motivated by an empirical measurement study of DNN inference on EC2 GPU instances. While existing works on guaranteeing inference performance service level objectives (SLOs) focus on either temporal sharing of GPUs or reactive GPU resource scaling and inference migration techniques, how to proactively mitigate such severe performance interference has received comparatively little attention. In this paper, we propose iGniter, an interference-aware GPU resource provisioning framework for cost-efficiently achieving predictable DNN inference in the cloud. iGniter is comprised of two key components: (1) a lightweight DNN inference performance model, which leverages the system and workload metrics that are practically accessible to capture the performance interference;(2) A cost-efficient GPU resource provisioning strategy that jointly optimizes the GPU resource allocation and adaptive batching based on our inference performance model, with the aim of achieving predictable performance of DNN inference workloads. We implement a prototype of iGniter based on the NVIDIA Triton inference server hosted on EC2 GPU instances. Extensive prototype experiments on four representative DNN models and datasets demonstrate that iGniter can guarantee the performance SLOs of DNN inference workloads with practically acceptable runtime overhead, while saving the monetary cost by up to 25% in comparison to the state-of-the-art GPU resource provisioning strategies. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

Model-enhanced vector index 23

学校读者我要写书评

暂无评论

Model-enhanced vector index

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Hailin Zhang Yujing Wang Qi Chen Ruiheng Chang Ting Zhang Ziming Miao Yingyan Hou Yang Ding Xupeng Miao Haonan Wang Bochen Pang Yuefeng Zhan Hao Sun Weiwei Deng Qi Zhang Fan Yang Xing Xie Mao Yang Bin Cui School of Computer Science & Key Lab of High Confidence Software Technologies Peking University and Microsoft Microsoft Microsoft and Aerospace Information Research Institute & Key Laboratory of Target Cognition and Application Technology Chinese Academy of Sciences Microsoft and Institute of Information Engineering Chinese Academy of Sciences Carnegie Mellon University National University of Singapore School of Computer Science & Key Lab of High Confidence Software Technologies Peking University and National Engineering Laboratory for Big Data Analysis and Applications Peking University

Embedding-based retrieval methods construct vector indices to search for document representations that are most similar to the query representations. They are widely used in document retrieval due to low latency and decent recall performance. Recent research indicates that deep retrieval solutions offer better model quality, but are hindered by unacceptable serving latency and the inability to support document updates. In this paper, we aim to enhance the vector index with end-to-end deep generative models, leveraging the differentiable advantages of deep retrieval models while maintaining desirable serving efficiency. We propose Model-enhanced Vector Index (MEVI), a differentiable model-enhanced index empowered by a twin-tower representation model. MEVI leverages a Residual Quantization (RQ) codebook to bridge the sequence-to-sequence deep retrieval and embedding-based models. To substantially reduce the inference time, instead of decoding the unique document ids in long sequential steps, we first generate some semantic virtual cluster ids of candidate documents in a small number of steps, and then leverage the well-adapted embedding vectors to further perform a fine-grained search for the relevant documents in the candidate virtual clusters. We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.

关键词：

How Does Knowledge Graph Embedding Extrapolate to Unseen data: A Semantic Evidence View

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Ren Cao, Yanan Zhu, Qiannan Bi, Guanqun Fang, Fang Liu, Yi Li, Qian Institute of Information Engineering Chinese Academy of Sciences China School of Cyber Security University of Chinese Academy of Sciences China Gaoling School of Artificial Intelligence Renmin University of China China Beijing Key Laboratory of Big Data Management and Analysis Methods China National Computer Network Emergency Response Technical Team Coordination Center of China China University of Technology Sydney Australia

Knowledge Graph Embedding (KGE) aims to learn representations for entities and relations. Most KGE models have gained great success, especially on extrapolation scenarios. Specifically, given an unseen triple (h, r, t), a trained model can still correctly predict t from (h, r,?), or h from (?, r, t), such extrapolation ability is impressive. However, most existing KGE works focus on the design of delicate triple modeling function, which mainly tells us how to measure the plausibility of observed triples, but offers limited explanation of why the methods can extrapolate to unseen data, and what are the important factors to help KGE extrapolate. Therefore in this work, we attempt to study the KGE extrapolation of two problems: 1. How does KGE extrapolate to unseen data? 2. How to design the KGE model with better extrapolation ability? For the problem 1, we first discuss the impact factors for extrapolation and from relation, entity and triple level respectively, propose three Semantic Evidences (SEs), which can be observed from train set and provide important semantic information for extrapolation. Then we verify the effectiveness of SEs through extensive experiments on several typical KGE methods. For the problem 2, to make better use of the three levels of SE, we propose a novel GNN-based KGE model, called Semantic Evidence aware Graph Neural Network (SE-GNN). In SE-GNN, each level of SE is modeled explicitly by the corresponding neighbor pattern, and merged sufficiently by the multi-layer aggregation, which contributes to obtaining more extrapolative knowledge representation. Finally, through extensive experiments on FB15k-237 and WN18RR datasets, we show that SE-GNN achieves state-of-the-art performance on Knowledge Graph Completion task and performs a better extrapolation ability. Our code is available at https://***/renli1024/SE-GNN. Copyright © 2021, The Authors. All rights reserved.

关键词： Knowledge graph

Cross-domain Human Parsing via Adversarial Feature and Label Adaptation

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Liu, Si Sun, Yao Zhu, Defa Ren, Guanghui Chen, Yu Feng, Jiashi Han, Jizhong Institute of Information Engineering Chinese Academy of Sciences *** Department of Ece National University of Singapore Jiangsu Key Laboratory of Big Data Analysis Technology/B-DAT Nanjing University of Information Science and Technology Collab. Innov. Ctr. of Atmosph. Environ. and Equip. Technol. Nanjing Univ. of Info. Sci. and Technol. Nanjing China

Human parsing has been extensively studied recently (Yamaguchi et al. 2012;Xia et al. 2017) due to its wide applications in many important scenarios. Mainstream fashion parsing models (i.e., parsers) focus on parsing the high-resolution and clean images. However, directly applying the parsers trained on benchmarks of high-quality samples to a particular application scenario in the wild, e.g., a canteen, airport or workplace, often gives non-satisfactory performance due to domain shift. In this paper, we explore a new and challenging cross-domain human parsing problem: Taking the benchmark dataset with extensive pixel-wise labeling as the source domain, how to obtain a satisfactory parser on a new target domain without requiring any additional manual labeling? To this end, we propose a novel and efficient crossdomain human parsing model to bridge the cross-domain differences in terms of visual appearance and environment conditions and fully exploit commonalities across domains. Our proposed model explicitly learns a feature compensation network, which is specialized for mitigating the cross-domain differences. A discriminative feature adversarial network is introduced to supervise the feature compensation to effectively reduces the discrepancy between feature distributions of two domains. Besides, our proposed model also introduces a structured label adversarial network to guide the parsing results of the target domain to follow the high-order relationships of the structured labels shared across domains. The proposed framework is end-to-end trainable, practical and scalable in real applications. Extensive experiments are conducted where LIP dataset is the source domain and 4 different datasets including surveillance videos, movies and runway shows without any annotations, are evaluated as target domains. The results consistently confirm data efficiency and performance advantages of the proposed method for the challenging cross-domain human parsing problem. Copyright

关键词： Security systems

A Wavelet-CNN-LSTM Model for Tailings Pond Risk Prediction

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Yang, Jun Li, Qing Sun, Yixuan Wang, Wei Li, Linchao Wang, Xuwei Jia, Shengyao Tong, Renyuan National and Local Joint Engineering research center for Disaster Monitoring Technologies and Instruments China Jiliang University Hangzhou310051 China Zhejiang Provincial Key Laboratory of Intelligent Manufacturing Quality Big Data Traceability Analysis and Application China Jiliang University Hangzhou310051 China Zhejiang PeckerAI Technology. Ltd Hangzhou310018 China College of Computer Science and Technology Zhejiang University Hangzhou310027 China

Tailings ponds are places for storing industrial waste. The saturation line is the key factor of quantifying the safety of tailings pond. Existing saturation line time-series prediction methods are mainly based on statistical models or shallow machine learning models. Although these models aim to capture the time dependence of the sequence data, the channel and temporal are even unavailable in principle. In this paper, we presents a two-stage forecasting method, which embeds the channel and temporal attention into a hybrid CNN-LSTM model to predict the saturation line. In the first stage, the discrete wavelet transform (DWT) is applied to capture the refined sequence information. In the second stage, the CNN-LSTM model is utilized to learn the basic spatial and temporal features in the time series. Furthermore, the channel and temporal attention model are embedded into CNN-LSTM model to enhance the feature extracting ability in channel and temporal dimension. Consequently, our proposed model is shown to outperform classic models on five real-world datasets. Copyright © 2020, The Authors. All rights reserved.

关键词： Forecasting

Model-assisted inference for covariate-specific treatment effects with high-dimensional data

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wu, Peng Tan, Zhiqiang Hu, Wenjie Zhou, Xiao-Hua Beijing International Center for Mathematical Research Peking University Beijing100871 China Department of Statistics Rutgers University 110 Frelinghuysen Road PiscatawayNJ08854 United States Department of Probability and Statistics Peking University Beijing100871 China Department of Biostatistics Beijing International Center for Mathematical Research National Engineering Laboratory of Big Data Analysis and Applied Technology Peking University Beijing100871 China

Covariate-specific treatment effects (CSTEs) represent heterogeneous treatment effects across subpopulations defined by certain selected covariates. In this article, we consider marginal structural models where CSTEs are linearly represented using a set of basis functions of the selected covariates. We develop a new approach in high-dimensional settings to obtain not only doubly robust point estimators of CSTEs, but also model-assisted confidence intervals, which are valid when a propensity score model is correctly specified but an outcome regression model may be misspecified. With a linear outcome model and subpopulations defined by discrete covariates, both point estimators and confidence intervals are doubly robust for CSTEs. In contrast, confidence intervals from existing high-dimensional methods are valid only when both the propensity score and outcome models are correctly specified. We establish asymptotic properties of the proposed point estimators and the associated confidence intervals. We present simulation studies and empirical applications which demonstrate the advantages of the proposed method compared with competing ones. Copyright © 2021, The Authors. All rights reserved.

关键词： Clustering algorithms

Hierarchical Attention Feature Fusion and Refinement Network for Point Cloud Upsampling

学校读者我要写书评

暂无评论

Hierarchical Attention Feature Fusion and Refinement Network...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Yaori Zhang Shujin Lin Fan Zhou Ruomei Wang National Engineering Research Center of Digital Life School of Computer Science and Engineering Guangdong Provincial Science and Technology Collaborative Innovation Center for Culture and Tourism Sun Yat-sen University Guangzhou China School of Communication and Design Guangdong Key Laboratory for Big Data Analysis and Simulation of Public Opinion Guangdong Provincial Science and Technology Collaborative Innovation Center for Culture and Tourism Sun Yat-sen University Guangzhou China Guangdong Provincial Science and Technology Collaborative Innovation Center for Culture and Tourism National Engineering Research Center of Digital Life School of Computer Science and Engineering Sun Yat-sen University Guangzhou China National Engineering Research Center of Digital Life School of Computer Science and Engineering Sun Yat-sen University Guangzhou China

ISBN: (数字)9798350390155

ISBN: (纸本)9798350390162

This paper presents a novel hierarchical attention feature fusion and refinement network designed to address challenges in existing deep learning based point cloud upsampling methods. The network combines self-attention layers with a multi-level feature extraction architecture, effectively integrating local and global features, thereby enhancing the robustness and uniformity of the point cloud. Furthermore, a spatial refinement module is employed to predict the offset between the generated coarse dense point clouds and real point clouds, thereby enhancing consistency with the ground truth. Concurrently, a filter function is applied in the loss function to handle outliers of generated point clouds. Extensive experimental results across multiple datasets indicate that our method outperforms existing approaches.

关键词： Point cloud compression Deep learning Feature extraction Robustness Encoding

COVID-19大流行期间流感活动呈“断崖式”下降——佩戴口罩、人员流动变化及SARS-CoV-2干扰的作用

学校读者我要写书评

暂无评论

engineering 2023年第2期21卷 195-202,M0008页

作者：韩莎莎张婷吕岩赖圣杰戴佩希郑建东杨维中周晓华冯录召 Beijing International Center for Mathematical Research Peking UniversityBeijing 100871China Harvard Medical School Harvard UniversityBostonMA 02115USA School of Population Medicine and Public Health Chinese Academy of Medical Sciences&Peking Union Medical CollegeBeijing 100730China Academy for Advanced Interdisciplinary Studies Peking UniversityBeijing 100871China WorldPop School of Geography and Environmental ScienceUniversity of SouthamptonSouthampton SO171BJUK Division for Infectious Diseases Chinese Center for Disease Control and PreventionBeijing 102206China Department of Epidemiology and Biostatistics School of Public HealthPeking UniversityBeijing 100871China National Engineering Laboratory of Big Data Analysis and Applied Technology Peking UniversityBeijing 100871China

一般情况下,每年冬季是季节性流感高发季节,但在当前2019冠状病毒病(COVID-19)大流行期间,全球季节性流感活动呈“断崖式”下降。为应对即将到来的流感季节,亟需弄清这种前所未有的流感低水平流行的原因。本文中,我们探索了一种国家特... 详细信息

一般情况下,每年冬季是季节性流感高发季节,但在当前2019冠状病毒病(COVID-19)大流行期间,全球季节性流感活动呈“断崖式”下降。为应对即将到来的流感季节,亟需弄清这种前所未有的流感低水平流行的原因。本文中,我们探索了一种国家特异性推理模型,利用该模型估计中国、英国和美国这三个国家中佩戴口罩、人员流动变化(国际和国内)以及严重急性呼吸综合征冠状病毒2型(SARS-CoV-2)干扰的影响。结果发现,在这些地区增加一周佩戴口罩的干预措施,流感活动可减少11.3%~35.2%。实施一周的限制人员流动措施对国际(1.7%~6.5%)和国内社区(1.6%~2.8%)的影响较小。2020年至2021年,仅佩戴口罩这一项干预措施就可使阳性率下降13.3%至19.8%。仅人员流动变化可使阳性率降低5.2%至14.0%,其中79.8%~98.2%归因于国际旅行限制。SARS-CoV-2干扰仅在2019年至2020年具有统计学显著效应。中国北方地区及英国的阳性百分比分别降低7.6%(2.4~14.4)和10.2%(7.2~13.6)。探索所得出的结果对了解非药物干预措施和其他呼吸系统疾病背景下的流感演变有一定意义,有助于制定卫生政策,并可为公共卫生措施的个性化设计提供信息。

关键词：卫生政策非药物干预呼吸系统疾病流行期间季节性流感流动变化个性化设计模型估计