检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Wu, Wei Pan, Zhuoshi Wang, Chao Chen, Liyi Bai, Yunchu Wang, Tianfu Fu, Kun Wang, Zheng Xiong, Hui University of Science and Technology of China China Tsinghua University China Alibaba Cloud Computing China Hong Kong University of Science and Technology Guangzhou China Hong Kong University of Science and Technology Hong Kong

The rapid advancement of Large Language Models (LLMs) has driven growing demand for processing extended context sequences in contemporary applications. However, this progress faces two major challenges: performance degradation due to sequence lengths out-of-distribution, and excessively long inference times caused by the quadratic computational complexity of attention. These issues hinder the application of LLMs in long-context scenarios. In this paper, we propose Dynamic Token-Level KV Cache Selection (TokenSelect), a training-free method for efficient and accurate long-context inference. TokenSelect builds upon the observation of non-contiguous attention sparsity, using Query-Key dot products to measure per-head KV Cache criticality at token-level. By per-head soft voting mechanism, TokenSelect selectively involves a few critical KV cache tokens in attention calculation without sacrificing accuracy. To further accelerate TokenSelect, we design the Selection Cache based on observations of consecutive Query similarity and implemented efficient dot product kernel, significantly reducing the overhead. A comprehensive evaluation of TokenSelect demonstrates up to 23.84× speedup in attention computation and up to 2.28× acceleration in end-to-end latency, while providing superior performance compared to state-of-the-art long-context inference methods. Copyright © 2024, The Authors. All rights reserved.

关键词： Cache memory

来源：评论

学校读者我要写书评

暂无评论

SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models

arXiv

引用

arXiv 2023年

作者： Wang, Feifei Tan, Zhentao Wei, Tianyi Wu, Yue Huang, Qidong University of Science and Technology of China China Alibaba Cloud

Despite the success of diffusion-based customization methods on visual content creation, increasing concerns have been raised about such techniques from both privacy and political perspectives. To tackle this issue, several anti-customization methods have been proposed in very recent months, predominantly grounded in adversarial attacks. Unfortunately, most of these methods adopt straightforward designs, such as end-to-end optimization with a focus on adversarially maximizing the original training loss, thereby neglecting nuanced internal properties intrinsic to the diffusion model, and even leading to ineffective optimization in some diffusion time steps. In this paper, we strive to bridge this gap by undertaking a comprehensive exploration of these inherent properties, to boost the performance of current anti-customization approaches. Two aspects of properties are investigated: 1) We examine the relationship between time step selection and the model’s perception in the frequency domain of images and find that lower time steps can give much more contributions to adversarial noises. This inspires us to propose an adaptive greedy search for optimal time steps that seamlessly integrates with existing anti-customization methods. 2) We scrutinize the roles of features at different layers during denoising and devise a sophisticated feature-based optimization framework for anti-customization. Experiments on facial benchmarks demonstrate that our approach significantly increases identity disruption, thereby protecting user privacy and copyright. Our code is available at: https://***/somuchtome/SimAC. Copyright © 2023, The Authors. All rights reserved.

关键词： Frequency domain analysis

来源：评论

学校读者我要写书评

暂无评论

Predictive Analysis of COVID-19 Data using Machine Learning

Predictive Analysis of COVID-19 Data using Machine Learning

引用

2023 Cross Strait Radio science and Wireless technology Conference, CSRSWTC 2023

作者： Gong, Bin Zhu, Yizhao Li, Mingchao Lv, Wei City University of Macau Faculty of Data Science Macau China School of Business Shenzhen Institute of Technology Shenzhen China Alibaba Cloud College of Big Data Application Zhuhai College of Science and Technology Zhuhai China

ISBN: (纸本)9798350358971

The COVID-19 pandemic has had far-reaching health consequences, including mental health issues and long-term physical symptoms. Mental health problems such as anxiety and depression have surged due to isolation and uncertainty. Concerns are rising regarding the pandemic's lasting impact on mental well-being. Furthermore, COVID-19 patients may experience persistent symptoms like fatigue and muscle weakness, affecting their long-term health. Leveraging machine learning (ML) techniques, and analyzing post-pandemic health impacts can reveal patterns and research insights. ML can utilize extensive data from electronic health records and public health databases to identify trends and potential future health issues. Additionally, ML models can monitor medical records and public health data, aiding in the swift detection and response to disease outbreaks by public health officials. © 2023 IEEE.

关键词： Health Analytics Machine Learning Prediction Recognizing Patterns

来源：评论

学校读者我要写书评

暂无评论

ROPO: Robust Preference Optimization for Large Language Models

arXiv

引用

arXiv 2024年

作者： Liang, Xize Chen, Chao Qiu, Shuang Wang, Jie Wu, Yue Fu, Zhihang Shi, Zhihao Wu, Feng Ye, Jieping University of Science and Technology of China China Alibaba Cloud China HKUST Hong Kong

Preference alignment is pivotal for empowering large language models (LLMs) to generate helpful and harmless responses. However, the performance of preference alignment is highly sensitive to the prevalent noise in the preference data. Recent efforts for this problem either marginally alleviate the impact of noise without the ability to actually reduce its presence, or rely on costly teacher LLMs prone to reward misgeneralization. To address these challenges, we propose the RObust Preference Optimization (ROPO) framework, an iterative alignment approach that integrates noise-tolerance and filtering of noisy samples without the aid of external models. Specifically, ROPO iteratively solves a constrained optimization problem, where we dynamically assign a quality-aware weight for each sample and constrain the sum of the weights to the number of samples we intend to retain. For noise-tolerant training and effective noise identification, we derive a robust loss by suppressing the gradients of samples with high uncertainty. We demonstrate both empirically and theoretically that the derived loss is critical for distinguishing noisy samples from clean ones. Furthermore, inspired by our derived loss, we propose a robustness-guided rejection sampling technique to compensate for the potential important information in discarded queries. Experiments on three widely-used datasets with Mistral-7B and Llama-2-7B demonstrate that ROPO significantly outperforms existing preference alignment methods, with its superiority growing as the noise rate increases. Copyright © 2024, The Authors. All rights reserved.

关键词： Alignment

来源：评论

学校读者我要写书评

暂无评论

Structure-aware Table-to-Text Generation with Prefix-tuning 23

Structure-aware Table-to-Text Generation with Prefix-tuning

引用

2023 4th International Conference on Control, Robotics and Intelligent System, CCRIS 2023

作者： Liu, Cencen Xu, Yi Yin, Wen Zheng, Dezhang University of Electronic Science and Technology of China Chengdu China Trusted Cloud Computing and Big Data Key Laboratory of Sichuan Province Sichuan Chengdu China

ISBN: (纸本)9798400708190

Table-to-text generation is designed to generate descriptive natural language for structured tables that conforms to objective facts and follows the source data. The current challenge in this field is to capture the structural information of the table and improve the quality of the generated text. The existing sequence-to-sequence approach is to linearize the table, which leads to miss captured structure information and is not conducive to the model learning contextual semantics. In this paper, we introduce structural-aware self-attention, which focuses on table structure to capture cell relationships between the same row or column. In this way, the generated descriptive text can more accurately reflect the correlation between the cells in the table, discarding irrelevant information. In order to adapt the pre-trained language model to the table-to-text generation task, we introduce prefix-tuning. Traditional fine-tuning methods update all model parameters, which leads to increased training costs. In contrast, using prefix-tuning for a more lightweight approach can improve model performance considerably. Attaching continuous prompts to tokens helps the model better understand the structure and semantics of the input sequence. All of our models are extended based on T5 and have strong competitiveness in the ToTTo dataset and Hitab dataset compared with several classical baselines. © 2023 ACM.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Pronounced advance on typhoon track forecast with global convection-permitting model

引用

science Bulletin 2025年第9期70卷 1424-1428页

作者： Gu, Jun Zhao, Chun Li, Gudongze Feng, Jiawang Xu, Mingyue Du, Qiuyan Xia, Zihan Li, Yubin Chen, Guanghua Hao, Xiaoyu Chen, Junshi An, Hong Deep Space Exploration Laboratory/School of Earth and Space Sciences/CMA-USTC Laboratory of Fengyun Remote Sensing/State Key Laboratory of Fire Science/Institute of Advanced Interdisciplinary Research on High-Performance Computing Systems and Software University of Science and Technology of China Hefei230026 China Laoshan Laboratory Qingdao266237 China CAS Center for Excellence in Comparative Planetology University of Science and Technology of China Hefei230026 China School of Atmospheric Physics Nanjing University of Information Science and Technology Nanjing210044 China Key Laboratory of Cloud-Precipitation Physics and Severe Storms Institute of Atmospheric Physics Chinese Academy of Sciences Beijing100029 China School of Computer Science and Technology University of Science and Technology of China Hefei230026 China

来源：评论

学校读者我要写书评

暂无评论

Local Context-Aware for Weakly Supervised Object Detection

Local Context-Aware for Weakly Supervised Object Detection

引用

2024 International Conference on Cyber-Physical Social Intelligence, ICCSI 2024

作者： Chen, Pengyu Nie, Xiushan Li, Tuo Bi, Yanchao Shandong Jianzhu University School of Computer Science and Technology Jinan China Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Co. Ltd. Jinan China

ISBN: (纸本)9798350376739

Weakly supervised object detection (WSOD) is designed to learn the class and location information of objects using image-level labels. Traditional WSOD approaches usually utilize Multiple-Instance Learning (MIL) to create numerous proposals and then choose the most distinctive ones as targets. However, methods that rely on convolutional neural networks tend to emphasize the most salient features of an object, often overlooking the complete object boundaries. In this paper, we propose a single-stage Local Context-Aware (LCA) based Transformer for WSOD. We employ category-wide patch mapping (CWPM) to take into account cross-patch information globally. We also incorporate the Local Awakening Module (LAM), which leverages local features to steer the model's learning process and enhance weak local responses. Comprehensive experiments on the VOC and COCO datasets showcase the efficacy of LCA. Meanwhile, LCA achieves much faster inference speed than the two-stage WSOD approaches. © 2024 IEEE.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

A Reduced State-Space Generation Method for Concurrent Systems Based on CPN-PR Model

引用

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 2025年第6期44卷 2328-2342页

作者： Zhong, Wenjie Sun, Tao Zhou, Jian-Tao Wang, Zhuowei Song, Xiaoyu Inner Mongolia University College of Computer Science the Engineering Research Center of Ecological Big Data Ministry of Education the Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software the Inner Mongolia Engineering Laboratory for Big Data Analysis Technology Hohhot010000 China Guangdong University of Technology School of Computer Science and Technology Guangzhou510006 China Portland State University Department of Electrical and Computer Engineering PortlandOR97207 United States

Colored Petri nets (CPNs) provide descriptions of the concurrent behaviors for software and hardware. Model checking based on CPNs is an effective method to simulate and verify the concurrent behavior in system design. However, the model-checking method traverses the full state space, which suffers from the state-space explosion problem. A reduced state-space generation method related to the property of concurrent systems is proposed. Specifically, we extend CPNs to define a property-related model (CPN-PR) and give a property-related analysis method whose results can be used to generate the CPN-PR model. A reduced state-space generation method is developed based on enabled binding element filtering rules. The stutter trace equivalence between the state spaces of CPN and CPN-PR has been proven by showing that the reduced state space may not change the model-checking result. A comparison experiment is conducted to demonstrate the effectiveness of our method. © 1982-2012 IEEE.

关键词： Model checking

来源：评论

学校读者我要写书评

暂无评论

Deep Time-Series Prediction of Complications and Deterioration in ICU Patients Based on the MSIPA Model

Deep Time-Series Prediction of Complications and Deteriorati...

引用

2023 Cross Strait Radio science and Wireless technology Conference, CSRSWTC 2023

作者： Feng, Duanyang Wang, Wennan Yan, Junjie Song, Shiyang Liu, Yiwen Guo, Xiaomin City University of Macau Faculty of Data Science Macau China Academy of Management Guangdong University of Science and Technology Dongguan China Alibaba Cloud Big Data Application College Zhuhai College of Science and Technology Zhuhai China Information Technology School Hebei University of Economics and Business Shijiazhuang China

ISBN: (纸本)9798350358971

In this research, a novel deep time-series model, MSIPA, is introduced for predicting complications and deterioration in ICU patients. Unlike conventional models, MSIPA efficiently handles data across varied time intervals, offering enhanced accuracy, particularly with discontinuous time series. It outperforms other deep learning models in multiple tasks, emphasizing the promise of deep time series prediction in healthcare. © 2023 IEEE.

关键词： Deep Temporal Prediction Disease Deterioration Forecasting MSIPA Model

来源：评论

学校读者我要写书评

暂无评论

computing Power Scheduling Method Based on Long-Term Workload Prediction for computing Power Platform 6

Computing Power Scheduling Method Based on Long-Term Workloa...

引用

6th International Conference on Communications, Information System and Computer Engineering, CISCE 2024

作者： Yang, Wangying Zhu, Guoping Chen, Qin Liu, Nianchao School of Computer Science Beijing University of Posts and Telecommunications Beijing China China Electronics Cloud Computing Technology Co. Ltd. Cloud Product R&d Department Wuhan China S&t Development China Electronics Corporation Shenzhen China

ISBN: (纸本)9798350353174

Users' requests for the computing power often exhibit suddenness and unpredictability, which creates challenges for workload scheduling of computing power platforms. Traditional scheduling methods struggle to adapt to such situations as they typically schedule based on real-time load conditions, ignoring long-term trends in load variations. To address these limitations, we propose a computing power scheduling method based on long-term workload prediction. First, we leverage a tree structure to aggregate temporal-spatial information from multi-variate operating time-series data, effectively describing the characteristics of workload on computing power platform. Building upon Informer, we employ residual mechanisms and dilated causal convolutions to extract deep features. Additionally, we integrate Wasserstein Generative Adversarial Network (WGAN) with Informer to accurately forecast future changes in workload for computing platforms, and introduce a novel training algorithm to enhance efficiency. We validate our method through experiments using real-world workload datasets, demonstrating the significant reduction in Mean Absolute Error (MAE) compared to LSTM. Furthermore, our model exhibits sensitivity to the length of encoder input sequences, indicating improved performance with longer series inputs. © 2024 IEEE.

关键词： computing power

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：