Large language models (LLMs) demonstrate emergent in-context learning capabilities, where they adapt to new tasks based on example demonstrations. However, in-context learning has seen limited effectiveness in many se...
详细信息
Large language models (LLMs) demonstrate emergent in-context learning capabilities, where they adapt to new tasks based on example demonstrations. However, in-context learning has seen limited effectiveness in many settings, is difficult to quantitatively control and takes up context window space. To overcome these limitations, we propose an alternative approach that recasts in-context learning as in-context vectors (ICV). Using ICV has two steps. We first use a forward pass on demonstration examples to create the in-context vector from the latent embedding of the LLM. This vector captures essential information about the intended task. On a new query, instead of adding demonstrations to the prompt, we shift the latent states of the LLM using the ICV. The ICV approach has several benefits: 1) it enables the LLM to more effectively follow the demonstration examples;2) it's easy to control by adjusting the magnitude of the ICV;3) it reduces the length of the prompt by removing the in-context demonstrations;4) ICV is computationally much more efficient than fine-tuning. We demonstrate that ICV achieves better performance compared to standard in-context learning and fine-tuning on diverse tasks including safety, style transfer, role-playing and formatting. Moreover, we show that we can flexibly teach LLM to simultaneously follow different types of instructions by simple vector arithmetics on the corresponding ICVs. Code is available at https://***/shengliu66/ICV. Copyright 2024 by the author(s)
Oja's algorithm for Streaming Principal Component Analysis (PCA) for n datapoints in a d dimensional space achieves the same sin-squared error O(reff/n) as the offline algorithm in O(d) space and O(nd) time and a ...
We consider the problem of learning temporal logic formulas from examples of system behavior. Learning temporal properties has crystallized as an effective means to explain complex temporal behaviors. Several efficien...
详细信息
Brain cancer can take many forms, but glioblastoma (GBM) is one of the most aggressive. To treat it effectively, doctors need to know the genetic subtype of a specific part of the tumor called the O-6-methylguanine-DN...
详细信息
Gaming communities are groups of people from different nations, religions, genders, and ages who come to play games and have discussions about the games. However, some gaming communities are plagued by toxic behavior ...
详细信息
Predicting how different interventions will causally affect a specific individual is important in a variety of domains such as personalized medicine, public policy, and online marketing. There are a large number of me...
详细信息
Current mobile applications(apps) have become increasingly complicated with increasing features that are represented on the graphical user interface associated with application programming interfaces(APIs) to access b...
详细信息
Current mobile applications(apps) have become increasingly complicated with increasing features that are represented on the graphical user interface associated with application programming interfaces(APIs) to access backend functionality and data. Meanwhile, apps suffer from the “software bloat” in volume. Some app features may be redundant, with respect to those features(from other apps) that the users already desirably and frequently use. However, the current app release model forces users to download and install a full-size installation package rather than optionally choosing only their desired features. Large-size apps can not only increase the local resource consumption, such as CPU, memory, and energy, but also inevitably compromise the user experience, such as the slow load time in the app. In this article, we first conduct an empirical study to characterize the app feature usage when users interact with Android apps,and surprisingly find that users access only a very small subset of app features. Based on these findings,we design a new approach named Lego Droid, which automatically decomposes an Android app for flexible loading and installation, while preserving the expected functionality with a fast and instant app load. With such a method, a slimmer bundle will be downloaded and host the target APIs inside the original app to satisfy users' requirements. We implement a system for Lego Droid and evaluate it with 1000 real-world Android apps. Compared to the original full-size apps, apps optimized by Lego Droid can substantially improve the load time by reducing the base bundle and feature bundles by 13.06% and 10.93%, respectively,along with the app-package installation size by 44.17%. In addition, we also demonstrate that Lego Droid is quite practical with evolving versions, as it can produce substantial reusable code to alleviate the developers' efforts when releasing new app versions.
The skin has a very significant role in humans. But unfortunately, the skin is one of the organs vulnerable to disease. Skin diseases can have an impact on the annoying itching, pain, emotional, and social feelings of...
详细信息
This paper presents a novel framework for creating a recoverable rare disease patient identity system using blockchain and smart contracts, decentralized identifiers (DIDs), and the InterPlanetary File System (IPFS). ...
详细信息
In recent years, many researchers have become interested in how the Metaverse can be applied in higher education. Although the use of Metaverse in education is still in its early stages, ongoing research on virtual wo...
详细信息
暂无评论