检索结果-内蒙古大学图书馆

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is available at this url. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques 38

Suppress Content Shift: Better Diffusion Features via Off-th...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Yang, Zhiyong Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Computer Science and Tech. University of Chinese Academy of Sciences China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are powerful generative models, and this capability can also be applied to discrimination. The inner activations of a pre-trained diffusion model can serve as features for discriminative tasks, namely, diffusion feature. We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. To be specific, there are content differences between features and the input image, such as the exact shape of a certain object. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. Further empirical study also indicates that its negative impact is not negligible even when content shift is not visually perceivable. Hence, we propose to suppress content shift to enhance the overall quality of diffusion features. Specifically, content shift is related to the information drift during the process of recovering an image from the noisy input, pointing out the possibility of turning off-the-shelf generation techniques into tools for content shift suppression. We further propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology. Despite the simplicity, the proposed approach has achieved superior results on various tasks and datasets, validating its potential as a generic booster for diffusion features. Our code is available at this url. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Neural matching models for question retrieval and next question prediction in conversation

arXiv

引用

arXiv 2017年

作者： Yang, Liu Zamani, Hamed Zhang, Yongfeng Guo, Jiafeng Bruce Croft, W. Center for Intelligent Information Retrieval University of Massachusetts Amherst AmherstMA United States CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

The recent boom of AI has seen the emergence of many human-computer conversation systems such as Google Assistant, Microsoft Cortana, Amazon Echo and Apple Siri. We introduce and formalize the task of predicting questions in conversations, where the goal is to predict the new question that the user will ask, given the past conversational context. This task can be modeled as a "sequence matching" problem, where two sequences are given and the aim is to learn a model that maps any pair of sequences to a matching probability. Neural matching models, which adopt deep neural networks to learn sequence representations and matching scores, have attracted immense research interests of information retrieval and natural language processing communities. In this paper, we first study neural matching models for the question retrieval task that has been widely explored in the literature, whereas the effectiveness of neural models for this task is relatively unstudied. We further evaluate the neural matching models in the next question prediction task in conversations. We have used the publicly available Quora data and Ubuntu chat logs in our experiments. Our evaluations investigate the potential of neural matching models with representation learning for question retrieval and next question prediction in conversations. Experimental results show that neural matching models perform well for both tasks. Copyright © 2017, The Authors. All rights reserved.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

Learning a deep listwise context model for ranking refinement

arXiv

引用

arXiv 2018年

作者： Ai, Qingyao Bi, Keping Guo, Jiafeng Croft, W. Bruce College of Information and Computer Sciences University of Massachusetts Amherst AmherstMA United States CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Learning to rank has been intensively studied and widely applied in information retrieval. Typically, a global ranking function is learned from a set of labeled data, which can achieve good performance on average but may be suboptimal for individual queries by ignoring the fact that relevant documents for different queries may have different distributions in the feature space. Inspired by the idea of pseudo relevance feedback where top ranked documents, which we refer as the local ranking context, can provide important information about the query's characteristics, we propose to use the inherent feature distributions of the top results to learn a Deep Listwise Context Model that helps us fine tune the initial ranked list. Specifically, we employ a recurrent neural network to sequentially encode the top results using their feature vectors, learn a local context model and use it to re-rank the top results. There are three merits with our model: (1) Our model can capture the local ranking context based on the complex interactions between top results using a deep neural network;(2) Our model can be built upon existing learning-to-rank methods by directly using their extracted feature vectors;(3) Our model is trained with an attention-based loss function, which is more effective and efficient than many existing listwise methods. Experimental results show that the proposed model can significantly improve the state-of-the-art learning to rank methods on benchmark retrieval corpora. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Graph wavelet neural network

arXiv

引用

arXiv 2019年

作者： Xu, Bingbing Shen, Huawei Cao, Qi Qiu, Yunqi Cheng, Xueqi CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences School of Computer and Control Engineering University of Chinese Academy of Sciences Beijing China

We present graph wavelet neural network (GWNN), a novel graph convolutional neural network (CNN), leveraging graph wavelet transform to address the shortcomings of previous spectral graph CNN methods that depend on graph Fourier transform. Different from graph Fourier transform, graph wavelet transform can be obtained via a fast algorithm without requiring matrix eigendecomposition with high computational cost. Moreover, graph wavelets are sparse and localized in vertex domain, offering high efficiency and good interpretability for graph convolution. The proposed GWNN significantly outperforms previous spectral graph CNNs in the task of graph-based semi-supervised classification on three benchmark datasets: Cora, Citeseer and Pubmed. Copyright © 2019, The Authors. All rights reserved.

关键词： Wavelet transforms

来源：评论

学校读者我要写书评

暂无评论

Optimization from Structured Samples for Coverage Functions

arXiv

引用

arXiv 2020年

作者： Chen, Wei Sun, Xiaoming Zhang, Jialin Zhang, Zhijie Microsoft Research Asia Beijing China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China

We revisit the optimization from samples (OPS) model, which studies the problem of optimizing objective functions directly from the sample data. Previous results showed that we cannot obtain a constant approximation ratio for the maximum coverage problem using polynomially many independent samples of the form {Si, f (Si)}ti=1 (Balkanski et al., 2017), even if coverage functions are (1 − ǫ)-PMAC learnable using these samples (Badanidiyuru et al., 2012), which means most of the function values can be approximately learned very well with high probability. In this work, to circumvent the impossibility result of OPS, we propose a stronger model called optimization from structured samples (OPSS) for coverage functions, where the data samples encode the structural information of the functions. We show that under three general assumptions on the sample distributions, we can design efficient OPSS algorithms that achieve a constant approximation for the maximum coverage problem. We further prove a constant lower bound under these assumptions, which is tight when not considering computational efficiency. Moreover, we also show that if we remove any one of the three assumptions, OPSS for the maximum coverage problem has no constant approximation. Copyright © 2020, The Authors. All rights reserved.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

Relative Importance Sampling for off-Policy Actor-Critic in Deep Reinforcement Learning

arXiv

引用

arXiv 2018年

作者： Humayoo, Mahammad Cheng, Xueqi CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100190 China

Off-policy learning is more unstable compared to on-policy learning in reinforcement learning (RL). One of the reasons for instability of off-policy learning is a discrepancy between target (π) and behavior (b) policy distribution. The discrepancy between π and b distribution can be alleviated by employing the smooth variant of importance sampling (IS), such as relative importance sampling (RIS).The RIS has parameter β ∈ [0, 1] that controls the smoothness. To cope with the instability of off-policy learning, we present the first relative importance sampling-off-policy actor-critic (RIS-off-PAC) model-free algorithms in RL. In our method, A network yields the target policy (actor), the value function (critic) assessing the current policy (π) using samples drawn from the behavior policy. We use action values generated from the behavior policy in reward function to train our algorithms rather than from the target policy. We also use deep neural networks to train both the actor and critic. We evaluated our algorithms on a number of OpenAI Gym problems and demonstrated better or comparable performance to several state-of-the-art RL baselines. Copyright © 2018, The Authors. All rights reserved.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

label-consistency based graph neural networks for semi-supervised node classification

arXiv

引用

arXiv 2020年

作者： Xu, Bingbing Huang, Junjie Hou, Liang Shen, Huawei Gao, Jinhua Cheng, Xueqi CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences China School of Computer and Control Engineering University of Chinese Academy of Sciences China

Graph neural networks (GNNs) achieve remarkable success in graph-based semi-supervised node classification, leveraging the information from neighboring nodes to improve the representation learning of target node. The success of GNNs at node classification depends on the assumption that connected nodes tend to have the same label. However, such an assumption does not always work, limiting the performance of GNNs at node classification. In this paper, we propose label-consistency based graph neural network (LC-GNN), leveraging node pairs unconnected but with the same labels to enlarge the receptive field of nodes in GNNs. Experiments on benchmark datasets demonstrate the proposed LC-GNN outperforms traditional GNNs in graph-based semi-supervised node classification. We further show the superiority of LC-GNN in sparse scenarios with only a handful of labeled nodes. Copyright © 2020, The Authors. All rights reserved.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Parameter estimation with the ordered 2 regularization via an alternating direction method of multipliers

arXiv

引用

arXiv 2019年

作者： Humayoo, Mahammad Cheng, Xueqi CAS Key Laboratory of Network Data Science & Technology Institute of Computing Technology Chinese Academy of Sciences Beijing 100190 China University of Chinese Academy of Sciences Beijing 100049 China

Regularization is a popular technique in machine learning for model estimation and for avoiding overfitting. Prior studies have found that modern ordered regularization can be more effective in handling highly correlated, high-dimensional data than traditional regularization. The reason stems from the fact that the ordered regularization can reject irrelevant variables and yield an accurate estimation of the parameters. How to scale up the ordered regularization problems when facing large-scale training data remains an unanswered question. This paper explores the problem of parameter estimation with the ordered 2-regularization via Alternating Direction Method of Multipliers (ADMM), called ADMM-O2. The advantages of ADMM-O2 include (i) scaling up the ordered 2 to a large-scale dataset, (ii) predicting parameters correctly by excluding irrelevant variables automatically, and (iii) having a fast convergence rate. Experimental results on both synthetic data and real data indicate that ADMM-O2 can perform better than or comparable to several state-of-the-art baselines. Copyright © 2019, The Authors. All rights reserved.

关键词： Parameter estimation

来源：评论

学校读者我要写书评

暂无评论

Iterative Channel Estimation for OTFS-based LEO-Sat Communication Using Comb-type ZC Sequences

Iterative Channel Estimation for OTFS-based LEO-Sat Communic...

引用

2024 IEEE Global Communications Conference, GLOBECOM 2024

作者： Shi, Haiwei Xing, Wang Zhou, Yiqing Zhang, Yu Zhou, Shuo Shi, Jinglin Institute of Computing Technology Cas State Key Lab of Processors Beijing China ShanghaiTech University School of Information Science and Technology Shanghai201210 China University of Chinese Academy of Sciences Beijing China Tsinghua University Department of Electronic Engineering Beijing100084 China

ISBN: (纸本)9798350351255

In low earth orbit satellite (LEO-Sat) communication systems based on orthogonal time frequency space (OTFS) modulation, existing channel estimation methods usually use high-power pilots, leading to a high peak-to-average power ratio (PAPR). In this paper, we propose a comb-type pilot based frame structure by inserting multiple identical Zadoff-Chu (ZC) sequences along the Doppler dimension. The ZC sequences are arranged at equal intervals in the delay dimension. The power of pilots is consistent with the power of data symbols, which can achieve low PAPR at the transmitter. At the receiver, based on the analyzed power distribution of the correlation function, we derive the power threshold to identify the received pilots. Then, an iterative ZC-sequences based channel estimation algorithm is proposed aided by data detection and interference cancellation. Through iterations, the data interference can be alleviated and estimation accuracy can be improved. Simulation results demonstrate that the proposed scheme can achieve superior channel estimation performance with low PAPR in the time domain. © 2024 IEEE.

关键词： Satellite communication systems

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：