检索结果-内蒙古大学图书馆

DDistill-SR: Reparameterized Dynamic Distillation Network for Lightweight Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Yan Su, Tongtong Li, Yusen Cao, Jiuwen Wang, Gang Liu, Xiaoguang Nankai-Baidu Joint Lab School of Computer Science Nankai University Tianjin300350 China Tianjin Key Laboratory of Network and Data Security Technology Tianjin300071 China Machine Learning and I-health International Cooperation Base of Zhejiang Province and Artificial Intelligence Institute Hangzhou Dianzi University 300018 China

Recent research on deep convolutional neural networks (CNNs) has provided a significant performance boost on efficient super-resolution (SR) tasks by trading off the performance and applicability. However, most existing methods focus on subtracting feature processing consumption to reduce the parameters and calculations without refining the immediate features, which leads to inadequate information in the restoration. In this paper, we propose a lightweight network termed DDistill- SR, which significantly improves the SR quality by capturing and reusing more helpful information in a static-dynamic feature distillation manner. Specifically, we propose a plug-in reparameterized dynamic unit (RDU) to promote the performance and inference cost trade-off. During the training phase, the RDU learns to linearly combine multiple reparameterizable blocks by analyzing varied input statistics to enhance layerlevel representation. In the inference phase, the RDU is equally converted to simple dynamic convolutions that explicitly capture robust dynamic and static feature maps. Then, the information distillation block is constructed by several RDUs to enforce hierarchical refinement and selective fusion of spatial context information. Furthermore, we propose a dynamic distillation fusion (DDF) module to enable dynamic signals aggregation and communication between hierarchical modules to further improve performance. Empirical results show that our DDistill- SR outperforms the baselines and achieves state-of-the-art results on most super-resolution domains with much fewer parameters and less computational overhead. We have released the code of DDistill-SR at https://***/icandle/DDistill-SR. Copyright © 2023, The Authors. All rights reserved.

关键词： Convolution

Your Transformer May Not be as Powerful as You Expect

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Luo, Shengjie Li, Shanda Zheng, Shuxin Liu, Tie-Yan Wang, Liwei He, Di Key Laboratory of Machine Perception MOE School of Intelligence Science and Technology Peking University China Machine Learning Department School of Computer Science Carnegie Mellon University United States Microsoft Research United States Center for Data Science Peking University China Zhejiang Lab China

Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far as we know, theoretical understanding of the RPE-based Transformers is largely unexplored. In this work, we mathematically analyze the power of RPE-based Transformers regarding whether the model is capable of approximating any continuous sequence-to-sequence functions. One may naturally assume the answer is in the affirmative-RPE-based Transformers are universal function approximators. However, we present a negative result by showing there exist continuous sequence-to-sequence functions that RPE-based Transformers cannot approximate no matter how deep and wide the neural network is. One key reason lies in that most RPEs are placed in the softmax attention that always generates a right stochastic matrix. This restricts the network from capturing positional information in the RPEs and limits its capacity. To overcome the problem and make the model more powerful, we first present sufficient conditions for RPE-based Transformers to achieve universal function approximation. With the theoretical guidance, we develop a novel attention module, called Universal RPE-based (URPE) Attention, which satisfies the conditions. Therefore, the corresponding URPE-based Transformers become universal function approximators. Extensive experiments covering typical architectures and tasks demonstrate that our model is parameter-efficient and can achieve superior performance to strong baselines in a wide range of applications. The code will be made publicly available at https://***/lsj2408/URPE. Copyright © 2022, The Authors. All rights reserved.

关键词： Stochastic systems

Contrastive Continual learning with Importance Sampling and Prototype-Instance Relation Distillation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Jiyong Azizov, Dilshod Li, Yang Liang, Shangsong School of Computer Science and Engineering Sun Yat-sen University China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Department of Machine Learning Mohamed bin Zayed University of Artificial Intelligence United Arab Emirates AI Thrust Information Hub The Hong Kong University of Science and Technology Guangzhou China Department of CSE The Hong Kong University of Science and Technology China

Recently, because of the high-quality representations of contrastive learning methods, rehearsal-based contrastive continual learning has been proposed to explore how to continually learn transferable representation embeddings to avoid the catastrophic forgetting issue in traditional continual settings. Based on this framework, we propose Contrastive Continual learning via Importance Sampling (CCLIS) to preserve knowledge by recovering previous data distributions with a new strategy for Replay Buffer Selection (RBS), which minimize estimated variance to save hard negative samples for representation learning with high quality. Furthermore, we present the Prototype-instance Relation Distillation (PRD) loss, a technique designed to maintain the relationship between prototypes and sample representations using a self-distillation process. Experiments on standard continual learning benchmarks reveal that our method notably outperforms existing baselines in terms of knowledge preservation and thereby effectively counteracts catastrophic forgetting in online contexts. The code is available at https://***/lijy373/CCLIS. Copyright © 2024, The Authors. All rights reserved.

关键词： Distillation

Cross-Store Next-Basket Recommendation

学校读者我要写书评

暂无评论

Cross-Store Next-Basket Recommendation

IEEE International Conference on data Mining (ICDM)

作者： Liang-Chen Ma Ya Li Zi-Feng Mai Fei-Yao Liang Chang-Dong Wang Min Chen Mohsen Guizani School of Computer Science and Engineering Sun Yat-sen University Guangzhou China School of Electronics and Information Guangdong Polytechnic Normal University Guangzhou China Guangdong Provincial Key Laboratory of Intellectual Property and Big Data Guangzhou China School of Computer Science and Engineering South China University of Technology Guangzhou China Pazhou Laboratory Guangzhou China Machine Learning Department Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) Abu Dhabi UAE

ISBN: (数字)9798331506681

ISBN: (纸本)9798331506698

Next-basket recommendation (NBR) infers a set of items that a user will interact with in the next basket. Existing methods often struggle with the data sparsity problem, particularly when the number of baskets is significantly large due to diverse user behaviors. Cross-domain recommendation (CDR) can effectively alleviate this problem in NBR by transferring knowledge across different domains. Nevertheless, these methods often rely on the similarities of overlapping users, which leads to the negative transfer problem and ignores the overlapping items that are general in real-world scenarios like chain stores. In this paper, we provide a clear symbolic definition of cross-store recommendation (CSR) and distinguish it from CDR. We also propose a novel CSNBR model for cross-store next-basket recommendation task. To fully model the transferable collaborative information between two stores, we learn the embeddings of users, baskets, and items by two intra-store bipartite graphs, and use an inter-store unified bipartite graph to transfer the previously learned knowledge. Furthermore, to alleviate the negative transfer problem, we propose to reconstruct the inter-store unified bipartite graph by utilizing user embeddings obtained from the transfer layer and the disentanglement layer. We also employ two sequence encoders to model the historical sequential information at basket-level and item-level. Extensive experiments conducted on real-world datasets demonstrate the effectiveness of the CSNBR model.

关键词： Neural networks Collaboration data models Bipartite graph data mining

RecCoder: Reformulating Sequential Recommendation as Large Language Model-Based Code Completion

学校读者我要写书评

暂无评论

RecCoder: Reformulating Sequential Recommendation as Large L...

IEEE International Conference on data Mining (ICDM)

作者： Kai-Huang Lai Wu-Dong Xi Xing-Xing Xing Wei Wan Chang-Dong Wang Min Chen Mohsen Guizani School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Contribution during internship at NetEase Games UX Center NetEase Games Guangzhou China Guangdong Provincial Key Laboratory of Intellectual Property and Big Data Guangzhou China School of Computer Science and Engineering South China University of Technology Guangzhou China Pazhou Laboratory Guangzhou China Machine Learning Department Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) Abu Dhabi UAE

ISBN: (数字)9798331506681

ISBN: (纸本)9798331506698

In the evolving landscape of sequential recommendation systems, the application of Large Language Models (LLMs) is increasingly prominent. However, current attempts typically utilize general-purpose LLMs, which present a mismatch in capability and a large semantic gap relative to the specialized needs of recommendation tasks. To tackle these issues, we introduce RecCoder, an innovative model that reformulates sequential recommendation as a code completion task. This approach leverages the superior reasoning capability of code LLMs as a backbone, aligning well with the requirements of recommendation systems. To bridge the semantic gap, RecCoder creates extra tokens for each item and employs item content to initialize token embeddings. Furthermore, we have developed a suite of Semantic Adaptation Fine-tuning tasks, tailored to enhance the model's acquisition of both content and collaborative semantic information, thus aligning the model's intrinsic capabilities with the unique demands of recommendation tasks. Through extensive testing on three public datasets, RecCoder has shown remarkable improvements over existing models in terms of recommendation accuracy and efficiency. This success highlights the substantial yet previously underexplored potential of code LLMs in improving recommendation accuracy and efficiency, suggesting a promising new direction for future research in this area. The implementation code is accessible at https://***/AllminerLab/Code-for-RecCoder-master.

关键词： Adaptation models Codes Accuracy Large language models Semantics Cognition data models data mining Recommender systems Testing

Contrastive learning for Adapting Language Model to Sequential Recommendation

学校读者我要写书评

暂无评论

Contrastive Learning for Adapting Language Model to Sequenti...

IEEE International Conference on data Mining (ICDM)

作者： Fei-Yao Liang Wu-Dong Xi Xing-Xing Xing Wei Wan Chang-Dong Wang Min Chen Mohsen Guizani School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Contribution during internship at NetEase Games UX Center NetEase Games Guangzhou China Guangdong Provincial Key Laboratory of Intellectual Property and Big Data Guangzhou China School of Computer Science and Engineering South China University of Technology Guangzhou China Pazhou Laboratory Guangzhou China Machine Learning Department Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI) Abu Dhabi UAE

ISBN: (数字)9798331506681

ISBN: (纸本)9798331506698

With the explosive growth of information, recommendation systems have emerged to alleviate the problem of information overload. In order to improve the performance of recommendation systems, many existing methods introduce Large Language Models to extract textual information from description text. However, Large Language Models are trained on large-scale generic textual data and may face a semantic gap for downstream recommendation tasks. To address the above issues, we propose Contrastive learning for Adapting Language Model to Sequential Recommendation (CLA-Rec). In CLA-Rec, we first extract text embeddings from description text using Large Language Models and align the text embeddings learned by Large Language Models with the collaborative information through contrastive learning to obtain high-quality item representations. Through semantic alignment, we bridge the semantic gap between Large Language Models and the recommendation task. To map textual information and collaborative information into user representations, we utilize a Transformer model to learn user representations and capture user preferences by combining the semantically aligned item representations. Extensive experiments on three public datasets demonstrate that our method outperforms state-of-the-art approaches on multiple evaluation metrics, illustrating the effectiveness of the CLA-Rec model in adapting Large Language Models to recommendation tasks.

关键词： Measurement Adaptation models Large language models Semantics Collaboration Contrastive learning Transformers data models data mining Recommender systems

Ndmfcs: An Automatic Fruit Counting System in Modern Apple Orchard Using Abatement of Abnormal Fruit Detection

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Wu, Zhenchao Sun, Xiaoming Jiang, Hanhui Mao, Wulan Li, Rui Andriyanov, Nikita Soloviev, Vladimir Fu, Longsheng College of Mechanical and Electronic Engineering Northwest A&F University Shaanxi Yangling712100 China Key Laboratory of Agricultural Internet of Things Ministry of Agriculture and Rural Affairs Shaanxi Yangling712100 China Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Service Shaanxi Yangling712100 China Northwest A&F University Shenzhen Research Institute Guangdong Shenzhen518000 China Department of Data Analysis and Machine Learning Financial University under the Government of the Russian Federation Moscow125167 Russia Institute of Agricultural Mechanization Xinjiang Academy of Agricultural Sciences Urumqi830000 China

Automatic fruit counting is an important task for growers to estimate yield and manage orchards. Although many deep-learning-based fruit detection algorithms have been developed to improve performance of automatic fruit counting systems, abnormal fruit detection has often been caused by these algorithms detecting non-target fruits that have similar growth characteristics to target fruits. Abnormal fruit detection in modern apple orchard, i.e. detected fruits in the back row of trees (DFBRT) or on the ground (DFG), would result in a higher number of fruits counting than the ground truth. This study proposes an automatic fruit counting system called NDMFCS (Normal Detection Matched Fruit Counting System) to solve this problem for improving fruit counting accuracy in modern apple orchard. NDMFCS consists of three sub-systems, i.e. object detection based on You Only Look Once Version 4-tiny (YOLOv4-tiny), abatement of abnormal fruit detection based on threshold, and fruit counting based on trunk tracking and ID assignment. YOLOv4-tiny was selected to implement detection of fruits and trunks, whose output is confidence and pixel coordinates of detected object. The DFBRT and the DFG were defined as abnormal fruit detection and abated by thresholds to improve detection performance of fruit. Detected fruits were removed when their distance from camera is further than a distance threshold (defined as the DFBRT) or the confidence of fruit detection is less than a confidence threshold (defined as the DFG). Finally, fruit counting was implemented by trunk tracking and ID assignment, where each fruit was assigned a unique tracking ID. Results on 10 sets of original videos indicated that average fruit detection precision was improved from 89.1% to 93.3% after abatement of abnormal fruit detection. Also, Multiple Object Tracking Accuracy and Multiple Object Tracking Precision were improved on average by 4.2% and 3.3%, respectively, while average ID Switch Rate was decreased on ave

关键词： Object detection

learning Physics-Informed Neural Networks without Stacked Back-propagation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： He, Di Li, Shanda Shi, Wenlei Gao, Xiaotian Zhang, Jia Bian, Jiang Wang, Liwei Liu, Tie-Yan National Key Laboratory of General Artificial Intelligence School of Intelligence Science and Technology Peking University China Machine Learning Department School of Computer Science Carnegie Mellon University United States Microsoft Research Center for Data Science Peking University China

Physics-Informed Neural Network (PINN) has become a commonly used machine learning approach to solve partial differential equations (PDE). But, facing high-dimensional second-order PDE problems, PINN will suffer from severe scalability issues since its loss includes second-order derivatives, the computational cost of which will grow along with the dimension during stacked back-propagation. In this work, we develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. We further discuss the model capacity and provide variance reduction methods to address key limitations in the derivative estimation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is significantly faster. Our code is released at https://***/LithiumDA/PINN-without-Stacked-BP. Copyright © 2022, The Authors. All rights reserved.

关键词： Backpropagation

Evaluating the Impact of Special Characters on Password Security: a Comparative Analysis

学校读者我要写书评

暂无评论

IEEE Network 2025年

作者： He, Daojing Liu, Zhiyong Chan, Sammy Zhou, Beibei Guizani, Mohsen Harbin Institute of Technology School of Computer Science and Technology Shenzhen China Guizhou University State Key Laboratory of Public Big Data Guizhou China Research and Development Center Guangzhou Jingyuan Security Technology Co. Ltd Guangzhou China City University of Hong Kong Kowloon Department of Electrical Engineering Hong Kong Hong Kong Shanghai Urban Construction Vocational College School of Artificial Intelligence Application Shanghai China Machine Learning Department Abu Dhabi United Arab Emirates

Text-based password authentication remains essential for identity verification on internet platforms and blockchain systems. However, as computational power advances, security threats to user passwords have intensified, raising concerns about their effectiveness. While special characters (i.e., non-alphanumeric symbols, which include punctuation marks, mathematical symbols, and other non-letter/number characters like '.', '@', '#', '$', and similar) are often recommended to enhance password strength, the National Institute of Standards and Technology (NIST) prioritizes length over complexity, challenging their necessity. This study fills a critical gap in understanding user behavior regarding special character usage. By analyzing 15 datasets from Chinese, English, and German languages, we examined patterns of special character incorporation and their impact on password strength. Our findings indicate that users frequently employ predictable methods for including special characters, potentially compromising security, especially under stringent policies. We present three key contributions: (1) an analysis of special character usage patterns, (2) a quantitative assessment of strength differences using the PCFGv4.1 model, and (3) actionable recommendations for stakeholders to enhance password security practices. This research further advocates for aligning password policies with user behavior for more effective security. © 1986-2012 IEEE.

关键词： Authentication Protocol