With the rapid advancement of big data and artificial intelligence technologies, the governance of sensitive data has become a critical and pressing issue. Traditional data de-identification approaches exhibit ineffic...
详细信息
With the emergence of multimodal large models, the problem of hallucination has been plaguing their development and deployment. How to reliably detect the presence of hallucinations inmLLMshas become an important issu...
详细信息
ISBN:
(纸本)9789819794423;9789819794430
With the emergence of multimodal large models, the problem of hallucination has been plaguing their development and deployment. How to reliably detect the presence of hallucinations inmLLMshas become an important issue. We propose UHDF, which replaces the closed-source models in it with open-source models by improving UniHD [15], and dramatically outperforms it. By optimizing the external information used in UniHD and achieving decoupling between different external information sources, we minimize the hallucinations introduced in pipeline, and thus improve the effectiveness of hallucinations detection. UHDF using the open-source model outperforms UniHD using the closed-source model (GPT-4v), achieving 86.6% (dev set)/85.3% (test set) on MacroF1 and achieved the first place in NLPCC2024 Shared Task 10 Track1 (Open Source). Our code and models are available at https://***/codetalker125/UHDF.
Large language models (LLMs) have become foundational to numerous naturallanguageprocessing tasks;however, decoding coherent and contextually relevant text remains a complex challenge. In openended generation, maxim...
详细信息
ISBN:
(纸本)9789819794331;9789819794348
Large language models (LLMs) have become foundational to numerous naturallanguageprocessing tasks;however, decoding coherent and contextually relevant text remains a complex challenge. In openended generation, maximizing probability is often not the appropriate objective, as with sampling methods, the continuation tends to be incoherent and repetitive in various degrees. We propose Merge Decoding, merging information in the shallow layer, such as sequential information, with the final task-specific layer, thereby generating coherent and rich text. MD works across three scales of the LLaMA family(7B, 13B, 30B), achieving higher quality text in open-ended text generation (Wiki-Text, WikiNews, BookCorpus) and enhancing reasoning capabilities in downstream tasks (Gsm8k, StrategyQA) https://***/YcChou/MergeDecoding.
Recently, large language models (LLMs) achieve remarkable success in domains beyond the traditional naturallanguageprocessing, and there is a growing interest in applying LLMs to more general domains like code gener...
详细信息
As the digital transformation of education continues to advance, the inefficiency and subjectivity of traditional manual scoring methods have become increasingly prominent. To address this issue, this study developed ...
详细信息
Recent advancements in large-scale pre-trained automatic speech recognition (ASR) foundation models (e.g., Whisper) have exhibited remarkable performance in speech processing tasks. But fine-tuning such models for low...
详细信息
ISBN:
(纸本)9789819794362;9789819794379
Recent advancements in large-scale pre-trained automatic speech recognition (ASR) foundation models (e.g., Whisper) have exhibited remarkable performance in speech processing tasks. But fine-tuning such models for low-resource languages can be computationally expensive and prone to overfitting. Prompting methods offer a solution by designing specific prompts in the inputs that guide the model's behavior for targeted tasks, facilitating parameter-efficient adaptation. This paper presents the first exploration of various prompt tuning methods and optimized strategies for low-resource ASR based on Whisper. Moreover, we propose a shallow integration method to utilize the advantage of deep prompt tuning and reparametrization. Extensive experiments on the Common Voice and FLEURS datasets show the competitive performance of prompt tuning compared to full fine-tuning and Lora with fewer trainable parameters. Notably, the shallow integration strategy yields impressive results, especially for small models.
Social media has become the primary source of information for individuals, yet much of this information remains unverified. The rise of generative artificial intelligence has further accelerated the creation of unveri...
详细信息
ISBN:
(纸本)9789819794393;9789819794409
Social media has become the primary source of information for individuals, yet much of this information remains unverified. The rise of generative artificial intelligence has further accelerated the creation of unverified content. Adaptive rumor resolution systems are imperative for maintaining information integrity and public trust. Traditional methods have relied on encoder-based frameworks to enhance rumor representation and propagation characteristics. However, these models are often small in scale and lack generalizability for unforeseen events. Recent advances in Large language Models show promise but are unreliable in discerning truth from falsehood. Our work leverages LLMs by creating a testbed for predicting unprecedented rumors and designing a retrieval-augmented framework that integrates historical knowledge and collective intelligence. Experiments on two real-world datasets demonstrate the effectiveness of our proposed framework.
This paper presents an implementation scheme that integrates low-code platforms with artificial intelligence (AI) technology to enhance the development process. By leveraging AI for automatic code generation, intellig...
详细信息
Dialogue Discourse Parsing aims to identify the discourse links and relations between utterances, which has attracted more interest in recent years. Previous studies either adopt local optimization to independently se...
详细信息
ISBN:
(纸本)9789819794300;9789819794317
Dialogue Discourse Parsing aims to identify the discourse links and relations between utterances, which has attracted more interest in recent years. Previous studies either adopt local optimization to independently select one parent for each utterance or use global optimization to directly get the tree representing the dialogue structure. However, the influence of these two optimization methods remains less explored. In this paper, we aim to systematically inspect their performance. Specifically, for local optimization, we use local loss during the training stage and a greedy strategy during the inference stage. For global optimization, We implement optimization of unlabeled and labeled trees by structured losses including Max-Margin and TreeCRF, and exploit Chu-Liu-Edmonds algorithm during the inference stage. Experiments shows that the performance of these two optimization methods is closely related to the characteristics of the dataset, and global optimization can reduce the burden of identifying long-range dependency relations.
The rise of voice interface applications has renewed interest in improving the robustness of spoken language understanding(SLU). Many advances have come from end-to-end speech-language joint training, such as inferrin...
详细信息
ISBN:
(纸本)9789819794362;9789819794379
The rise of voice interface applications has renewed interest in improving the robustness of spoken language understanding(SLU). Many advances have come from end-to-end speech-language joint training, such as inferring semantics directly from speech signals and post-editing automatic speech recognition (ASR) output. Despite their performance achievements, these methods either suffer from the unavailability of a large number of paired error-prone ASR transcriptions and ground-truth annotations or are computationally costly. To mitigate these issues, we propose an ASR-robust pre-trained language model (ASRLM), which involves a generator generating simulated ASR transcriptions from ground-truth annotations and a sample-efficient discriminator distinguishing reasonable ASR errors from unrealistic ones. Experimental results demonstrate that ASRLM improves performance on a wide range of SLU tasks in the presence of ASR errors while saving 27% of the computation cost compared to baselines. Analysis also shows that our proposed generator is better than other simulation methods, including both BERT and GPT4-based, at simulating real-world ASR error situations.
暂无评论