检索结果-内蒙古大学图书馆

Sample Design Engineering: An Empirical Study of What Makes Good Downstream Fine-Tuning Samples for LLMs

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guo, Biyang Wang, He Xiao, Wenyilin Chen, Hong Lee, Zhuxin Han, Songqiao Huang, Hailiang AI Lab SIME Shanghai University of Finance and Economics China Ant Group China Guangdong Yunxi Technology China Key Laboratory of Interdisciplinary Research of Computation and Economics Ministry of Education China

In the burgeoning field of Large Language Models (LLMs) like ChatGPT and LLaMA, Prompt Engineering (PE) is renowned for boosting zero-shot or in-context learning (ICL) through prompt modifications. Yet, the realm of the sample design for downstream fine-tuning, crucial for task-specific LLM adaptation, is largely unexplored. This paper introduces Sample Design Engineering (SDE), a methodical approach to enhancing LLMs' post-tuning performance by refining input, output, and reasoning designs. We conduct a series of in-domain (ID) and out-of-domain (OOD) experiments to assess the impact of various design options on LLMs' downstream performance, revealing several intriguing patterns that hold consistently across different LLMs. Based on these insights, we propose an integrated SDE strategy, combining the most effective options, and validate its consistent superiority over heuristic sample designs in complex downstream tasks like multi-aspect sentiment analysis, event extraction, and nested entity recognition. Additionally, analyses of LLMs' inherent prompt/output perplexity, zero-shot, and ICL abilities illustrate that good PE strategies may not always translate to good SDE strategies. Code available at https://***/beyondguo/LLM-Tuning. Copyright © 2024, The Authors. All rights reserved.

关键词： Sentiment analysis

ADGym: design choices for deep anomaly detection 23

学校读者我要写书评

暂无评论

ADGym: design choices for deep anomaly detection

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Minqi Jiang Chaochuan Hou Ao Zheng Songqiao Han Hailiang Huang Qingsong Wen Xiyang Hu Yue Zhao AI Lab Shanghai University of Finance and Economics AI Lab Shanghai University of Finance and Economics and MoE Key Laboratory of Interdisciplinary Research of Computation and Economics DAMO Academy Alibaba Group Carnegie Mellon University

Deep learning (DL) techniques have recently found success in anomaly detection (AD) across various fields such as finance, medical services, and cloud computing. However, most of the current research tends to view deep AD algorithms as a whole, without dissecting the contributions of individual design choices like loss functions and network architectures. This view tends to diminish the value of preliminary steps like data preprocessing, as more attention is given to newly designed loss functions, network architectures, and learning paradigms. In this paper, we aim to bridge this gap by asking two key questions: (i) Which design choices in deep AD methods are crucial for detecting anomalies? (ii) How can we automatically select the optimal design choices for a given AD dataset, instead of relying on generic, preexisting solutions? To address these questions, we introduce ADGym, a platform specifically crafted for comprehensive evaluation and automatic selection of AD design elements in deep methods. Our extensive experiments reveal that relying solely on existing leading methods is not sufficient. In contrast, models developed using ADGym significantly surpass current state-of-the-art techniques.

关键词：

Sample-Based Matroid Prophet Inequalities

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fu, Hu Lu, Pinyan Tang, Zhihao Gavin Wu, Hongxun Wu, Jinzhao Zhang, Qianfan ITCS Key Laboratory of Interdisciplinary Research of Computation and Economics Shanghai University of Finance and Economics China UC Berkeley United States Yale University United States Princeton University United States

We study matroid prophet inequalities when distributions are unknown and accessible only through samples. While single-sample prophet inequalities for special matroids are known, no constant-factor competitive algorithm with even a sublinear number of samples was known for general matroids. Adding more to the stake, the single-sample version of the question for general matroids has close (two-way) connections with the long-standing matroid secretary conjecture. In this work, we give a (1/4 − Ε)-competitive matroid prophet inequality with only OΕ (poly log n) samples. Our algorithm consists of two parts: (i) a novel quantile-based reduction from matroid prophet inequalities to online contention resolution schemes (OCRSs) with OΕ (log n) samples, and (ii) a (1/4 − Ε)-selectable matroid OCRS with OΕ (poly log n) samples which carefully addresses an adaptivity challenge. Copyright © 2024, The Authors. All rights reserved.

关键词： Combinatorial mathematics

Strategic Facility Location via Predictions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Qingyun Gravin, Nick Im, Sungjin University of California 5200 N. Lake Road MercedCA95344 United States Key Laboratory of Interdisciplinary Research of Computation and Economics Shanghai University of Finance and Economics Shanghai China

The facility location with strategic agents is a canonical problem in the literature on mechanism design without money. Recently, Agrawal et. al. [ABG+22] considered this problem in the context of machine learning augmented algorithms, where the mechanism designer is also given a prediction of the optimal facility location. An ideal mechanism in this framework produces an outcome that is close to the social optimum when the prediction is accurate (consistency) and gracefully degrades as the prediction deviates from the truth, while retaining some of the worst-case approximation guarantees (robustness). The previous work only addressed this problem in the two-dimensional Euclidean space providing optimal trade-offs between robustness and consistency guarantees for deterministic mechanisms. We consider the problem for general metric spaces. Our only assumption is that the metric is continuous, meaning that any pair of points must be connected by a continuous shortest path. We introduce a novel mechanism that in addition to agents’ reported locations takes a predicted optimal facility location ô. We call this mechanism Harmonic, as it selects one of the reported locations ∼i with probability inversely proportional to d(ô, ∼i) + ∆ for a constant parameter ∆. While Harmonic mechanism is not truthful, we can characterize the set of undominated strategies for each agent i as solely consisting of the points on a shortest path from their true location i to the predicted location ô. We further derive consistency and robustness guarantees on the Price of Anarchy (PoA) for the game induced by the mechanism. Specifically, assuming that ∆ = Ε/2n · ∑i∈[n] d(ô, i) is closely related to the average distance to the predicted location ô, our consistency guarantee is arbitrarily close to optimum (PoA is 1 + Ε), while having a constant robustness guarantee (PoA is O(1+1/Ε3) in general and is O(1+1/Ε2) for strictly convex metric spaces). We also show that for a constant number of agents

关键词： Harmonic analysis

Driving forces and their relative contributions to hydrocarbon expulsion from deep source rocks: A case of the Cambrian source rocks in the Tarim Basin

学校读者我要写书评

暂无评论

Petroleum Science 2023年第1期20卷 20-33页

作者： Bo Pang Jun-Qing Chen Xiong-Qi Pang Tao Hu Yue Sheng State Key Laboratory of Petroleum Resources and Prospecting China University of PetroleumBeijing102249China College of Geosciences China University of PetroleumBeijing102249China College of Science China University of PetroleumBeijing102249China Laboratory of Optical Detection Technology for Oil and Gas China University of PetroleumBeijing102249China Basic Research Center for Energy Interdisciplinary China University of PetroleumBeijing102249China CNPC Economics&Technology Research Institute Beijing100029China

To thoroughly understand the dynamic mechanism of hydrocarbon expulsion from deep source rocks,in this study,five types of hydrocarbon expulsion dynamics(thermal expansion,hydrocarbon diffusion,compaction,product volume expansion,and capillary pressure difference(CPD))are studied.A model is proposed herein to evaluate the relative contribution of different dynamics for hydrocarbon expulsion using the principle of mass balance,and the model has been applied to the Cambrian source rocks in the Tarim *** evaluation results show that during hydrocarbon expulsion from the source rocks,the relative contribution of CPD is the largest(>50%),followed by compaction(10%-40%),product volume expansion(5%-30%),and thermal expansion(2%-20%).The relative contribution of diffusion to hydrocarbon expulsion is minimal(<10%).These results demonstrate that CPD plays an important role in the hydrocarbon expulsion process of deep source *** hydrocarbon expulsion process of source rocks can be categorized into three stages based on the contribution of different dynamics to the process:the first stage is dominated by compaction and diffusion to expel hydrocarbons,the second stage is dominated by product volume expansion and CPD,and the third stage is dominated by product volume expansion and *** research offers new insights into hydrocarbon exploration in tight oil and gas reservoirs.

关键词： Driving force Dynamic mechanism Hydrocarbon expulsion Deep oil and gas exploration Tarim Basin

Last-iterate Convergence Separation between Extra-gradient and Optimism in Constrained Periodic Games

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Feng, Yi Li, Ping Panageas, Ioannis Wang, Xiao Institute for Theoretical Computer Science Shanghai University of Finance and Economics China Department of Computer Science University of California Irvine United States Key Laboratory of Interdisciplinary Research of Computation and Economics Ministry of Education China

Last-iterate behaviors of learning algorithms in repeated two-player zero-sum games have been extensively studied due to their wide applications in machine learning and related tasks. Typical algorithms that exhibit the last-iterate convergence property include optimistic and extragradient methods. However, most existing results establish these properties under the assumption that the game is time-independent. Recently, [Feng et al., 2023] studied the last-iterate behaviors of optimistic and extra-gradient methods in games with a time-varying payoff matrix, and proved that in an unconstrained periodic game, extra-gradient method converges to the equilibrium while optimistic method diverges. This finding challenges the conventional wisdom that these two methods are expected to behave similarly as they do in time-independent games. However, compared to unconstrained games, games with constrains are more common both in practical and theoretical studies. In this paper, we investigate the last-iterate behaviors of optimistic and extra-gradient methods in the constrained periodic games, demonstrating that similar separation results for last-iterate convergence also hold in this setting. © 2024, CC BY.

关键词： Gradient methods

Last-iterate convergence separation between extra-gradient and optimisim in constrained periodic games 24

学校读者我要写书评

暂无评论

Last-iterate convergence separation between extra-gradient a...

Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence

作者： Yi Feng Ping Li Ioannis Panageas Xiao Wang Institute for Theoretical Computer Science Shanghai University of Finance and Economics Department of Computer Science University of California Irvine Institute for Theoretical Computer Science Shanghai University of Finance and Economics and Key Laboratory of Interdisciplinary Research of Computation and Economics Ministry of Education

关键词：

Sample-Based Matroid Prophet Inequalities 24

学校读者我要写书评

暂无评论

Sample-Based Matroid Prophet Inequalities

Proceedings of the 25th ACM Conference on economics and computation

作者： Hu Fu Pinyan Lu Zhihao Gavin Tang Hongxun Wu Jinzhao Wu Qianfan Zhang Key Laboratory of Interdisciplinary Research of Computation and Economics and Institute for Theoretical Computer Science Shanghai University of Finance and Economics Shanghai China UC Berkeley Berkeley USA Yale University New Haven USA Princeton University Princeton USA

ISBN: (纸本)9798400707049

The classical prophet inequalities problem introduced by Krengel and Sucheston [1977, 1978] assumed complete knowledge of distributions. However, such an assumption may be unrealistic both in practice and for some *** study matroid prophet inequalities when distributions are unknown and accessible only through samples. While single-sample prophet inequalities for special matroids are known, no constant-factor competitive algorithm with even a sublinear number of samples was known for general matroids. Adding more to the stake, the single-sample version of the question for general matroids has close (two-way) connections with the long-standing matroid secretary *** this work, we give a (1/4 - ε)-competitive matroid prophet inequality with only Oε (poly log n) samples. Our algorithm consists of two parts: (i) a novel quantile-based reduction from matroid prophet inequalities to online contention resolution schemes (OCRSs) with Oε(log n) samples, and (ii) a (1/4 - ε)-selectable matroid OCRS with Oε (poly log n) samples which carefully addresses an adaptivity challenge.A full version of this paper can be found at https://***/abs/2406.12799.

关键词： prophet inequalities

ADGym: Design Choices for Deep Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Jiang, Minqi Hou, Chaochuan Zheng, Ao Han, Songqiao Huang, Hailiang Wen, Qingsong Hu, Xiyang Zhao, Yue AI Lab Shanghai University of Finance and Economics China MoE Key Laboratory of Interdisciplinary Research of Computation and Economics DAMO Academy Alibaba Group China Carnegie Mellon University United States University of Southern California United States

关键词： Anomaly detection