检索结果-内蒙古大学图书馆

Convergence analysis of nonconvex distributed stochastic zeroth-order coordinate method

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Shengjun Dong, Yunlong Xie, Dong Yao, Lisha Bailey, Colleen P. Fu, Shengli OSCAR Laboratory Department of Electrical Engineering University of North Texas DentonTX76207 United States School of Artificial Intelligence and Automation The MOE Key Laboratory of Image Processing and Intelligent Control The State Key Laboratory of Digital Manufacturing Equipment and Technology Huazhong University of Science and Technology Wuhan430074 China

This paper investigates the stochastic distributed nonconvex optimization problem of minimizing a global cost function formed by the summation of n local cost functions. We solve such a problem by involving zeroth-order (ZO) information exchange. In this paper, we propose a ZO distributed primal–dual coordinate method (ZODIAC) to solve the stochastic optimization problem. Agents approximate their own local stochastic ZO oracle along with coordinates with an adaptive smoothing parameter. We show that the proposed algorithm achieves the convergence rate of O(√p/√T) for general nonconvex cost functions. We demonstrate the efficiency of proposed algorithms through a numerical example in comparison with the existing state-of-the-art centralized and distributed ZO algorithms. Copyright © 2021, The Authors. All rights reserved.

关键词： Stochastic systems

Sample Weighted Gradient Regularization for Source-Free Domain Adaptation

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Long, Yunchong Xu, Bingrong Zhang, Qingyong Yin, Jianhua Zeng, Zhigang School of Automation Wuhan University of Technology Wuhan430070 China Intelligent Transportation Systems Research Center Wuhan University of Technology Wuhan430063 China The Engineering Research Center of Transportation Information and Safety Ministry of Education Wuhan430063 China School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China The Key Laboratory of Image Processing and Intelligent Control Education Ministry of China Wuhan430074 China

Conventional Domain Adaptation(DA) algorithms typically rely on access to the source data,which may not always be practical due to privacy concerns or limited storage *** address these challenges,Source-Free Domain Adaptation(SFDA) has emerged as an alternative that relies solely on a pre-trained source model rather than the actual source *** SFDA,pseudo-label supervised learning is commonly employed and initially plays a significant role in enhancing model performance during the early stages of target domain ***,as adaptation progresses,the performance can stagnate or even deteriorate due to the noise inherent in pseudo-labels,a phenomenon known as Confirmation *** noise present in target domain pseudo-labels introduces additional uncertainty and overfitting risks to the *** this study,a novel framework named Sample Weighted Gradient Regularization (SWGR) was introduced to tackle this *** first propose the Label Uncertainty Estimation Metric (LUEM),a reweighting strategy that assesses the reliability of *** metric evaluates label uncertainty from both the source domain pre-trained model and spatial distribution of target domain samples,allowing the model to achieve improved cross-domain transfer performance with high-confidence samples in the target domain. Additionally,we impose a constraint on the gradient descent process of the total loss function in the form of an inequality to prevent entropy from *** Gradient Regularization eliminates overfitting while also guiding the model in lowering prediction uncertainty *** approach reduces the uncertainty in the target domain and enhances the model's generalization *** results demonstrate that our method surpasses competing SFDA approaches and even outperforms many traditional DA methods which require extensive amounts of source domain data. © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

A Born-Again Takagi-Sugeno-Kang Fuzzy Classifier with Decoupled Fuzzy Dark Knowledge Distillation

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Zhang, Xiongtao Yin, Zezong Jiang, Yunliang Jiang, Yizhang Sun, Danfeng Liu, Yong School of Information Engineering Huzhou University Huzhou31300 China Zhejiang Key Laboratory of Intelligent Education Technology and Application Zhejiang Normal University Jinhua321004 China School of Computer Science and Technology Zhejiang Normal University Jinhua321004 China School of Artificial Intelligence and Computer Science Jiangnan University Wuxi 214122 China Key Laboratory of Image Processing and Intelligent Control Huazhong University of Science and Technology Ministry of Education Wuhan430074 China School of Computer Science Hangzhou Dianzi University Hangzhou310018 China College of Control Science and Engineering Zhejiang University Hangzhou310027 China

In order to enpower the hight performance as well as interpretability of low-order TSK fuzzy classifier, a born-again TSK fuzzy classier embedded with decoupled fuzzy dark knowledge distillation called HTSK-LLM-DKD is proposed in this study. HTSK-LLM DKD achieves the following distinctive characteristics: 1) It takes high-order TSK as teacher model and low-order TSK fuzzy classifier as student model, and leverages the proposed Least Learning Machine based Decoupling Knowledge Distillation to distill the fuzzy dark knowledge from high-order TSK fuzzy classifier to low-order TSK fuzzy classifier, which resulting in low-order TSK fuzzy classifier endowed with enhanced performance surpassing or at least comparable to high-order TSK fuzzy classifier;specifically 2) The Negative Euclidean distance between the output of teacher model and each class is employed to obtain the teacher logits, and then it computes teacher/student soft labels by the softmax function with distilling temperature parameter;3) By reformulating the Kullback Leibler divergence, it decouples fuzzy dark knowledge into target class knowledge and non-target class knowledge, and transfers them to student model, which taking HTSK-LLM-DKD with outstanding performance and relatively fast running speed. Extensive experiment results on UCI benchmarking datasets and real-world dataset Cleveland Heart Disease demonstrate the effectiveness of the proposed HTSK-LLM-DKD. © 2024, The Authors. All rights reserved.

关键词： Students

Exploiting Distilled Learning for Deep Siamese Tracking

学校读者我要写书评

暂无评论

Exploiting Distilled Learning for Deep Siamese Tracking

International Conference on Pattern Recognition

作者： Chengxin Liu Zhiguo Cao Wei Li Yang Xiao Shuaiyuan Du Angfan Zhu Key Laboratory of Image Processing and Intelligent Control Ministry of Education School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan China Queen Mary University of London

Existing deep siamese trackers are typically built on off-the-shelf CNN models for feature learning, with the demand for huge power consumption and memory storage. This limits current deep siamese trackers to be carried on resource-constrained devices like mobile phones, given factor that such a deployment normally requires cost-effective considerations. In this work, we address this issue by presenting a novel Distilled Learning Framework(DLF) for siamese tracking, which aims at learning tracking model with efficiency and high accuracy. Specifically, we propose two simple yet effective knowledge distillation strategies, denote as point-wise distillation and pairwise distillation, which are designed for transferring knowledge from a more discriminative teacher tracker into a compact student tracker. In this way, cost-effective and high performance tracking could be achieved. Extensive experiments on several tracking benchmarks demonstrate the effectiveness of our proposed method.

关键词： Power demand Pipelines Memory management Benchmark testing Mobile handsets Pattern recognition

A Stronger Baseline for Ego-Centric Action Detection

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Qing, Zhiwu Huang, Ziyuan Wang, Xiang Feng, Yutong Zhang, Shiwei Jiang, Jianwen Tang, Mingqian Gao, Changxin Ang, Marcelo H. Sang, Nong Key Laboratory of Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology ARC National University of Singapore Tsinghua University Alibaba Group

This technical report analyzes an egocentric video action detection method we used in the 2021 EPIC-KITCHENS-100 competition hosted in CVPR2021 Workshop. The goal of our task is to locate the start time and the end time of the action in the long untrimmed video, and predict action category. We adopt sliding window strategy to generate proposals, which can better adapt to short-duration actions. In addition, we show that classification and proposals are conflict in the same network. The separation of the two tasks boost the detection performance with high efficiency. By simply employing these strategy, we achieved 16.10% performance on the test set of EPIC-KITCHENS-100 Action Detection challenge using a single model, surpassing the baseline method by 11.7% in terms of average mAP. Copyright © 2021, The Authors. All rights reserved.

关键词： Machine learning

Convergence Analysis of Nonconvex Distributed Stochastic Zeroth-order Coordinate Method

学校读者我要写书评

暂无评论

Convergence Analysis of Nonconvex Distributed Stochastic Zer...

IEEE Conference on Decision and control

作者： Shengjun Zhang Yunlong Dong Dong Xie Lisha Yao Colleen P. Bailey Shengli Fu OS-CAR Laboratory University of North Texas Denton TX USA School of Artificial Intelligence and Automation the MOE Key Laboratory of Image Processing and Intelligent Control and the State Key Laboratory of Digital Manufacturing Equipment and Technology Huazhong University of Science and Technology Wuhan China University of North Texas Denton TX USA

ISBN: (纸本)9781665436601

This paper investigates the stochastic distributed nonconvex optimization problem of minimizing a global cost function formed by the summation of n local cost functions. We solve such a problem by involving zeroth-order (ZO) information exchange. In this paper, we propose a ZO distributed primal–dual coordinate method (ZODIAC) to solve the stochastic optimization problem. Agents approximate their own local stochastic ZO oracle along with coordinates under an adaptive smoothing parameter. We show that the proposed algorithm achieves the convergence rate of ${\mathcal{O}}(\sqrt p /\sqrt T )$ for general nonconvex cost functions. We demonstrate the efficiency of proposed algorithms through a numerical example in comparison with the existing state-of-the-art centralized and distributed ZO algorithms.

关键词： Smoothing methods Conferences Benchmark testing Cost function Approximation algorithms Information exchange Convergence

Continuous Cross-resolution Remote Sensing image Change Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Hao Zhang, Haotian Chen, keyan Zhou, Chenyao Chen, Song Zou, Zhengxia Shi, Zhenwei The Image Processing Center School of Astronautics Beihang University Beijing100191 China The Beijing Key Laboratory of Digital Media Beihang University Beijing100191 China The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China The Shanghai Artificial Intelligence Laboratory Shanghai200232 China The Department of Journalism and Communications Jeonuk National University Jeonju-Si54896 Korea Republic of The Department of Guidance Navigation and Control School of Astronautics Beihang University Beijing100191 China

Most contemporary supervised Remote Sensing (RS) image Change Detection (CD) approaches are customized for equal-resolution bitemporal images. Real-world applications raise the need for cross-resolution change detection, aka, CD based on bitemporal images with different spatial resolutions. Given training samples of a fixed bitemporal resolution difference (ratio) between the high-resolution (HR) image and the low-resolution (LR) one, current cross-resolution methods may fit a certain ratio but lack adaptation to other resolution differences. Toward continuous cross-resolution CD, we propose scale-invariant learning to enforce the model consistently predicting HR results given synthesized samples of varying resolution differences. Concretely, we synthesize blurred versions of the HR image by random downsampled reconstructions to reduce the gap between HR and LR images. We introduce coordinate-based representations to decode per-pixel predictions by feeding the coordinate query and corresponding multi-level embedding features into an MLP that implicitly learns the shape of land cover changes, therefore benefiting recognizing blurred objects in the LR image. Moreover, considering that spatial resolution mainly affects the local textures, we apply local-window self-attention to align bitemporal features during the early stages of the encoder. Extensive experiments on two synthesized and one real-world different-resolution CD datasets verify the effectiveness of the proposed method. Our method significantly outperforms several vanilla CD methods and two cross-resolution CD methods on the three datasets both in in-distribution and out-of-distribution settings. The empirical results suggest that our method could yield relatively consistent HR change predictions regardless of varying bitemporal resolution ratios. Our code is available at https://***/justchenhao/SILI_CD. Copyright © 2023, The Authors. All rights reserved.

关键词： Change detection

High Information Density and Low Coverage Data Storage in DNA with Efficient Channel Coding Schemes

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ding, Yi He, Xuan Nguyen, Tuan Thanh Song, Wentu Yakhini, Zohar Yaakobi, Eitan Pan, Linqiang Tang, Xiaohu Cai, Kui Information Coding and Transmission Key Lab of Sichuan Province Southwest Jiaotong University Sichuan Chengdu611756 China Cluster Singapore University of Technology and Design 487372 Singapore Faculty of Computer Science Technion - Israel Institute of Technology Haifa3200003 Israel School of Computer Science RUNI Herzliya4615200 Israel Key Laboratory of Image Information Processing and Intelligent Control of Education Ministry of China School of Artificial Intelligence and Automation Huazhong University of Science and Technology Hubei Wuhan430074 China

DNA-based data storage has been attracting significant attention due to its extremely high data storage density, low power consumption, and long duration compared to conventional data storage media. Despite the recent advancements in DNA data storage technology, significant challenges remain. In particular, various types of errors can occur during the processes of DNA synthesis, storage, and sequencing, including substitution errors, insertion errors, and deletion errors. Furthermore, the entire oligo may be lost. In this work, we report a DNA-based data storage architecture that incorporates efficient channel coding schemes, including different types of error-correcting codes (ECCs) and constrained codes, for both the inner coding and outer coding for the DNA data storage channel. We also carried out large scale experiments to validate our proposed DNA-based data storage architecture. Specifically, 1.61 and 1.69 MB data were encoded into 30,000 oligos each, with information densities of 1.731 and 1.815, respectively. It has been found that the stored information can be fully recovered without any error at average coverages of 4.5 and 6.0, respectively. This experiment achieved the highest net information density and lowest coverage among existing DNA-based data storage experiments (with standard DNA), with data recovery rates and coverage approaching theoretical optima. Copyright © 2024, The Authors. All rights reserved.

关键词： Channel coding

An advanced actor-critic algorithm for training video game ai 1st

学校读者我要写书评

暂无评论

An advanced actor-critic algorithm for training video game a...

1st International Conference on Neural Computing for Advanced Applications, NCAA 2020

作者： Zha, Zhong Yi Tang, Xue Song Wang, Bo Key Laboratory of Ministry of Education for Image Processing and Intelligent Control Artificial Intelligence and Automation School Huazhong University of Science and Technology Wuhan430074 China College of Information and Science Donghua University Shanghai China

ISBN: (纸本)9789811576690

This paper presents an improved deep reinforcement learning (DRL) algorithm, namely Advanced Actor Critic (AAC), which is based on Actor Critic (AC) algorithm, to the video game Artificial intelligence (AI) training. The advantage distribution estimator, Normal Constraint (NC) function and exploration based on confidence are introduced to improve the AC algorithm, in order to achieve accurate value estimate, select continuous spatial action correctly, and explorate effectively. The aim is to improve the performance of conventional AC algorithm in a complex environment. A case study of video game StarCraft II mini-game AI training is employed. The results verify that the improved algorithm effectively improves the performance in terms of the convergence rate, maximum reward, average reward in every 100 episode, and time to reach a specific reward, etc. The analysis on how these modifications improve the performance is also given through interpretation of the feature layers in the mini-games. © Springer Nature Singapore Pte Ltd. 2020.

关键词： Reinforcement learning