Model compression is one of the most popular approaches to improve the accessibility of Large Language Models (LLMs) by reducing their memory footprint. However, the gaining of such efficiency benefits often simultane...
详细信息
Model compression is one of the most popular approaches to improve the accessibility of Large Language Models (LLMs) by reducing their memory footprint. However, the gaining of such efficiency benefits often simultaneously demands extensive engineering efforts and intricate designs to mitigate the performance decline. In this work, we leverage (Soft) Prompt Tuning in its most vanilla form and discover such conventionally learned soft prompts can recover the performance of compressed LLMs. More surprisingly, we observe such recovery effect to be transferable among different tasks and models (albeit natural tokenizer and dimensionality limitations), resulting in further overhead reduction and yet, subverting the common belief that learned soft prompts are task-specific. Our work is fully orthogonal and compatible with model compression frameworks such as pruning and quantization, where we enable up to 8× compressed LLM (with a joint 4-bit quantization and 50% weight pruning compression) to match its uncompressed counterparts on popular benchmarks. We note that we are the first to reveal vanilla Parameter-Efficient Fine-Tuning (PEFT) techniques have the potential to be utilized under a compression recovery context, opening a new line of opportunities for model accessibility advancement while freeing our fellow researchers from the previously present engineering burdens and constraints. The code is available at https://***/zirui-ray-liu/compress-thenprompt. Copyright 2024 by the author(s)
Decentralized identification is an interesting topic for Internet-based systems. Although the use of centralized systems for identification is prevalent, there is still a need for decentralized identification systems ...
详细信息
Due to the characteristics of high resolution and rich texture information,visible light images are widely used for maritime ship ***,these images are suscep-tible to sea fog and ships of different sizes,which can res...
详细信息
Due to the characteristics of high resolution and rich texture information,visible light images are widely used for maritime ship ***,these images are suscep-tible to sea fog and ships of different sizes,which can result in missed detections and false alarms,ultimately resulting in lower detection *** address these issues,a novel multi-granularity feature enhancement network,MFENet,which includes a three-way dehazing module(3WDM)and a multi-granularity feature enhancement module(MFEM)is *** 3WDM eliminates sea fog interference by using an image clarity automatic classification algorithm based on three-way decisions and FFA-Net to obtain clear image ***,the MFEM improves the accuracy of detecting ships of different sizes by utilising an improved super-resolution reconstruction con-volutional neural network to enhance the resolution and semantic representation capa-bility of the feature maps from *** results demonstrate that MFENet surpasses the other 15 competing models in terms of the mean Average Pre-cision metric on two benchmark datasets,achieving 96.28%on the McShips dataset and 97.71%on the SeaShips dataset.
Extensive efforts have been made in designing large multiple-input multiple-output(MIMO)arrays. Nevertheless, improvements in conventional antenna characteristics cannot ensure significant MIMO performance improvement...
详细信息
Extensive efforts have been made in designing large multiple-input multiple-output(MIMO)arrays. Nevertheless, improvements in conventional antenna characteristics cannot ensure significant MIMO performance improvement in realistic multipath environments. Array decorrelation techniques have been proposed, achieving correlation reductions by either tilting the antenna beams or shifting the phase centers away from each other. Hence, these methods are mainly limited to MIMO terminals with small arrays. To avoid such problems, this work proposes a decorrelation optimization technique based on phase correcting surface(PCS)that can be applied to large MIMO arrays, enhancing their MIMO performances in a realistic(non-isotropic)multipath environment. First, by using a near-field channel model and an optimization algorithm, a near-field phase distribution improving the MIMO capacity is obtained. Then the PCS(consisting of square elements)is used to cover the array's aperture, achieving the desired near-field phase *** examples demonstrate the effectiveness of this PCS-based near-field optimization technique. One is a1 × 4 dual-polarized patch array(working at 2.4 GHz)covered by a 2 × 4 PCS with 0.6λ center-to-center distance. The other is a 2 × 8 dual-polarized dipole array, for which a 4 × 8 PCS with 0.4λ center-to-center distance is designed. Their MIMO capacities can be effectively enhanced by 8% and 10% in single-cell and multi-cell scenarios, respectively. The PCS has insignificant effects on mutual coupling, matching, and the average radiation efficiency of the patch array, and increases the antenna gain by about 2.5 dB while keeping broadside radiations to ensure good cellular coverage, which benefits the MIMO performance of the *** proposed technique offers a new perspective for improving large MIMO arrays in realistic multipath in a statistical sense.
In recent years,intelligent robots are extensively applied in the field of the industry and intelligent rehabilitation,wherein the human-robot interaction(HRI)control strategy is a momentous part that needs to be ***,...
详细信息
In recent years,intelligent robots are extensively applied in the field of the industry and intelligent rehabilitation,wherein the human-robot interaction(HRI)control strategy is a momentous part that needs to be ***,the efficacy and robustness of the HRI control algorithm in the presence of unknown external disturbances deserve to be *** deal with these urgent issues,in this study,artificial systems,computational experiments and a parallel execution intelligent control framework are constructed for the HRI *** upper limb-robotic exoskeleton system is re-modelled as an artificial *** on surface electromyogram-based subject's active motion intention in the practical system,a non-convex function activated anti-disturbance zeroing neurodynamic(NC-ADZND)controller is devised in the artificial system for parallel interaction and HRI control with the practical ***,the linear activation function-based zeroing neurodynamic(LAF-ZND)controller and proportionalderivative(posterior deltoid(PD))controller are presented and *** results substantiate the global convergence and robustness of the proposed controller in the presence of different external *** addition,the simulation results verify that the NC-ADZND controller is better than the LAF-ZND and the PD controllers in respect of convergence order and anti-disturbance characteristics.
Machine-to-machine (M2M) communication networks consist of resource-constrained autonomous devices, also known as autonomous Internet of things (IoTs) or machine-type communication devices (MTCDs) which act as a backb...
详细信息
Machine-to-machine (M2M) communication networks consist of resource-constrained autonomous devices, also known as autonomous Internet of things (IoTs) or machine-type communication devices (MTCDs) which act as a backbone for Industrial IoT, smart cities, and other autonomous systems. Due to the limited computing and memory capacity, these devices cannot maintain strong security if conventional security methods are applied such as heavy encryption. This article proposed a novel lightweight mutual authentication scheme including elliptic curve cryptography (ECC) driven end-to-end encryption through curve25519 such as (i): efficient end-to-end encrypted communication with pre-calculation strategy using curve25519;and (ii): elliptic curve Diffie-Hellman (ECDH) based mutual authentication technique through a novel lightweight hash function. The proposed scheme attempts to efficiently counter all known perception layer security threats. Moreover, the pre-calculated key generation strategy resulted in cost-effective encryption with 192-bit curve security. It showed comparative efficiency in key strength, and curve strength compared with similar authentication schemes in terms of computational and memory cost, communication performance and encryption robustness.
This paper presents a novel approach to noninvasive hyperglycemia monitoring utilizing electrocardiograms (ECG) from an extensive database comprising 1119 subjects. Previous research on hyperglycemia or glucose detect...
详细信息
Majority of contemporary fully supervised segmentation algorithms excel in quantifying human anatomy, significantly advancing the field. However, the reliance of numerous deep neural networks on extensive datasets wit...
详细信息
This article investigates large batch training techniques using layer-wise adaptive scaling ratio (LARS) across diverse settings. In particular, we first show that a state-of-the-art technique, called LARS with the wa...
详细信息
This study focuses on designing of lead-free double perovskite solar cells (DPSCs). Lead-free organic–inorganic DPSCs have achieved very good efficiency within a short period of active research. Formamidinium based d...
详细信息
暂无评论