检索结果-内蒙古大学图书馆

2024 Findings of the Association for Computational Linguistics: NAACL 2024

作者： Fu, Honghao Zhang, Liang Fu, Biao Zhao, Rui Su, Jinsong Shi, Xiaodong Chen, Yidong Institute of Artificial Intelligence Xiamen University China School of Informatics Xiamen University China Ministry of Culture and Tourism China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China

ISBN: (纸本)9798891761193

The primary objective of sign language translation (SLT) is to transform sign language videos into natural sentences. A crucial challenge in this field is developing signer-independent SLT systems which requires models to generalize effectively to signers not encountered during training. This challenge is exacerbated by the limited diversity of signers in existing SLT datasets, which often results in suboptimal generalization capabilities of current models. Achieving robustness to unseen signers is essential for signer-independent SLT. However, most existing method relies on signer identity labels, which is often impractical and costly in real-world applications. To address this issue, we propose the Signer Diversity-driven Data Augmentation (SDDA) method that can achieve good generalization without relying on signer identity labels. SDDA comprises two data augmentation schemes. The first is data augmentation based on adversarial training, which aims to utilize the gradients of the model to generate adversarial examples. The second is data augmentation based on diffusion model, which focuses on using the advanced diffusion-based text guided image editing method to modify the appearances of the signer in images. The combination of the two strategies significantly enriches the diversity of signers in the training process. Moreover, we introduce a consistency loss and a discrimination loss to enhance the learning of signer-independent features. Our experimental results demonstrate our model significantly enhances the performance of SLT in the signer-independent setting, achieving state-of-the-art results without relying on signer identity labels. © 2024 Association for Computational Linguistics.

关键词： Translation (languages)

来源：评论

学校读者我要写书评

暂无评论

Outlier-Aware Slicing for Post-Training Quantization in Vision Transformer 41

Outlier-Aware Slicing for Post-Training Quantization in Visi...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Ma, Yuexiao Li, Huixia Zheng, Xiawu Ling, Feng Xiao, Xuefeng Wang, Rui Wen, Shilei Chao, Fei Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University 361005 China ByteDance Inc. China Peng Cheng Laboratory Shenzhen China Institute of Artificial Intelligence Xiamen University China

Post-Training Quantization (PTQ) is a vital technique for network compression and acceleration, gaining prominence as model sizes increase. This paper addresses a critical challenge in PTQ: the severe impact of outliers on the accuracy of quantized transformer architectures. Specifically, we introduce the concept of 'reconstruction granularity' as a novel solution to this issue, which has been overlooked in previous works. Our work provides theoretical insights into the role of reconstruction granularity in mitigating the outlier problem in transformer models. This theoretical framework is supported by empirical analysis, demonstrating that varying reconstruction granularities significantly influence quantization performance. Our findings indicate that different architectural designs necessitate distinct optimal reconstruction granularities. For instance, the multi-stage Swin Transformer architecture benefits from finer granularity, a deviation from the trends observed in ViT and DeiT models. We further develop an algorithm for determining the optimal reconstruction granularity for various ViT models, achieving state-of-the-art (SOTA) performance in PTQ. For example, applying our method to 4-bit quantization, the Swin-Base model achieves a Top-1 accuracy of 82.24% on the ImageNet classification task. This result surpasses the RepQ-ViT by 3.92% (82.24% VS 78.32%). Similarly, our approach elevates the ViT-Small to a Top-1 accuracy of 80.50%, outperforming NoisyQuant by 3.64% (80.50% VS 76.86%). Copyright 2024 by the author(s)

关键词：

来源：评论

学校读者我要写书评

暂无评论

Grounded Chain-of-Thought for Multimodal Large Language Models

arXiv

引用

arXiv 2025年

作者： Wu, Qiong Yang, Xiangcong Zhou, Yiyi Fang, Chenxin Song, Baiyang Sun, Xiaoshuai Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University 361005 China

Despite great progress, existing multimodal large language models (MLLMs) are prone to visual hallucination, greatly impeding their trustworthy applications. In this paper, we study this problem from the perspective of visual-spatial reasoning, and propose a new learning task for MLLMs, termed Grounded Chain-of-Thought (GCoT). Different from recent visual CoT studies, which focus more on visual knowledge reasoning, GCoT is keen to helping MLLMs to recognize and ground the relevant visual cues step by step, thereby predicting the correct answer with grounding coordinates as the intuitive basis. To facilitate this task, we also carefully design and construct a dataset called multimodal grounded chain-of-thought (MM-GCoT) consisting of 24,022 GCoT examples for 5,033 images. Besides, a comprehensive consistency evaluation system is also introduced, including the metrics of answer accuracy , grounding accuracy and answer-grounding consistency. We further design and conduct a bunch of experiments on 12 advanced MLLMs, and reveal some notable findings: i. most MLLMs performs poorly on the consistency evaluation, indicating obvious visual hallucination;ii., visual hallucination is not directly related to the parameter size and general multimodal performance, i.e., a larger and stronger MLLM is not less affected by this issue. Lastly, we also demonstrate that the proposed dataset can help existing MLLMs to well cultivate their GCoT capability and reduce the inconsistent answering significantly. Moreover, their GCoT can be also generalized to exiting multimodal tasks, such as open-world QA and REC. Our dataset and evaluation scripts are anonymously released at: https://***/DoubtedSteam/MM-GCoT Copyright © 2025, The Authors. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Can LLMs Replace Clinical Doctors? Exploring Bias in Disease Diagnosis by Large Language Models

Can LLMs Replace Clinical Doctors? Exploring Bias in Disease...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Zhao, Yutian Wang, Huimin Liu, Yuqi Suhuang, Wu Wu, Xian Zheng, Yefeng Jarvis Research Center Tencent YouTu Lab Shenzhen China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China JMedical Artificial Intelligence Lab Westlake University Hangzhou China

ISBN: (纸本)9798891761681

The bias of disease prediction in Large Language Models (LLMs) is a critical yet underexplored issue, with potential implications for healthcare outcomes and equity. As LLMs increasingly find applications in healthcare, understanding and addressing their biases becomes paramount. This study focuses on this crucial topic, investigating the bias of disease prediction in models such as GPT-4, ChatGPT, and Qwen1.5-72b across gender, age range, and disease judgment behaviors.1 Utilizing a comprehensive real-clinical health record dataset of over 330,000 entries, we uncover that all three models exhibit distinct biases, indicating a pervasive issue of unfairness. To measure this, we introduce a novel metric-the diagnosis bias score, which reflects the ratio of prediction numbers to label numbers. Our in-depth analysis, based on this score, sheds light on the inherent biases in these models. In response to these findings, we propose a simple yet effective prompt-based solution to alleviate the observed bias in disease prediction with LLMs. This research underscores the importance of fairness in AI, particularly in healthcare applications, and offers a practical approach to enhance the equity of disease prediction models. © 2024 Association for Computational Linguistics.

关键词： Prediction models

来源：评论

学校读者我要写书评

暂无评论

AFFINEQUANT: AFFINE TRANSFORMATION QUANTIZATION FOR LARGE LANGUAGE MODELS 12

AFFINEQUANT: AFFINE TRANSFORMATION QUANTIZATION FOR LARGE LA...

引用

12th International Conference on Learning Representations, ICLR 2024

The significant resource requirements associated with Large-scale Language Models (LLMs) have generated considerable interest in the development of techniques aimed at compressing and accelerating neural networks. Among these techniques, Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its noteworthy compression efficiency and cost-effectiveness in the context of training. Existing PTQ methods for LLMs limit the optimization scope to scaling transformations between pre- and post-quantization weights. This constraint results in significant errors after quantization, particularly in low-bit configurations. In this paper, we advocate for the direct optimization using equivalent Affine transformations in PTQ (AffineQuant). This approach extends the optimization scope and thus significantly minimizing quantization errors. Additionally, by employing the corresponding inverse matrix, we can ensure equivalence between the pre- and post-quantization outputs of PTQ, thereby maintaining its efficiency and generalization capabilities. To ensure the invertibility of the transformation during optimization, we further introduce a gradual mask optimization method. This method initially focuses on optimizing the diagonal elements and gradually extends to the other elements. Such an approach aligns with the Levy-Desplanques theorem, theoretically ensuring invertibility of the transformation. As a result, significant performance improvements are evident across different LLMs on diverse datasets. Notably, these improvements are most pronounced when using very low-bit quantization, enabling the deployment of large models on edge devices. To illustrate, we attain a C4 perplexity of 15.76 (2.26↓ vs 18.02 in OmniQuant) on the LLaMA2-7B model of W4A4 quantization without overhead. On zero-shot tasks, AffineQuant achieves an average of 58.61% accuracy (1.98% ↑ vs 56.63 in OmniQuant) when using 4/4-bit quantization for LLaMA-30B, which setting a new st

关键词： Cost effectiveness

来源：评论

学校读者我要写书评

暂无评论

HRSAM: efficient Interactive Segmentation in High-Resolution Images

arXiv

引用

arXiv 2024年

作者： Huang, You Lai, Wenbin Ji, Jiayi Cao, Liujuan Zhang, Shengchuan Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China

The Segment Anything Model (SAM) has advanced interactive segmentation but is limited by the high computational cost on high-resolution images. This requires downsampling to meet GPU constraints, sacrificing the fine-grained details needed for high-precision interactive segmentation. To address SAM’s limitations, we focus on visual length extrapolation and propose a lightweight model named HRSAM. The extrapolation enables HRSAM trained on low resolutions to generalize to high resolutions. We begin by finding the link between the extrapolation and attention scores, which leads us to base HRSAM on Swin attention. We then introduce the Flexible Local Attention (FLA) framework, using CUDA-optimized efficient Memory Attention to accelerate HRSAM. Within FLA, we implement Flash Swin attention, achieving over a 35% speedup compared to traditional Swin attention, and propose a KV-only padding mechanism to enhance extrapolation. We also develop the Cycle-scan module that uses State Space models to efficiently expand HRSAM’s receptive field. We further develop the HRSAM++ within FLA by adding an anchor map, providing multi-scale data augmentation for the extrapolation and a larger receptive field at slight computational cost. Experiments show that, under standard training, HRSAMs surpass the previous SOTA with only 38% of the latency. With SAM-distillation, the extrapolation enables HRSAMs to outperform the teacher model at lower latency. Further finetuning achieves performance significantly exceeding the previous SOTA. Code is available at https://***/YouHuang67/*** © 2024, CC BY.

关键词： Extrapolation

来源：评论

学校读者我要写书评

暂无评论

Breaking the Bias: Recalibrating the Attention of Industrial Anomaly Detection

arXiv

引用

arXiv 2024年

作者： Chen, Xin Cao, Liujuan Zhang, Shengchuan Zheng, Xiewu Zhang, Yan Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China

Due to the scarcity and unpredictable nature of defect samples, industrial anomaly detection (IAD) predominantly employs unsupervised learning. However, all unsupervised IAD methods face a common challenge: the inherent bias in normal samples, which causes models to focus on variable regions while overlooking potential defects in invariant areas. To effectively overcome this, it is essential to decompose and recalibrate attention, guiding the model to suppress irrelevant variations and concentrate on subtle, defect-susceptible areas. In this paper, we propose Recalibrating Attention of Industrial Anomaly Detection (RAAD), a framework that systematically decomposes and recalibrates attention maps. RAAD employs a two-stage process: first, it reduces attention bias through quantization, and second, it fine-tunes defect-prone regions for improved sensitivity. Central to this framework is Hierarchical Quantization Scoring (HQS), which dynamically allocates bit-widths across layers based on their anomaly detection contributions. HQS dynamically adjusts bit-widths based on the hierarchical nature of attention maps, compressing lower layers that produce coarse and noisy attention while preserving deeper layers with sharper, defect-focused attention. This approach optimizes both computational efficiency and the model’s sensitivity to anomalies. We validate the effectiveness of RAAD on 32 datasets using a single 3090ti. Experiments demonstrate that RAAD, balances the complexity and expressive power of the model, enhancing its anomaly detection capability. Copyright © 2024, The Authors. All rights reserved.

关键词： Computational efficiency

来源：评论

学校读者我要写书评

暂无评论

VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models

arXiv

引用

arXiv 2024年

作者： Zhou, Chenyu Zhang, Mengdan Chen, Peixian Fu, Chaoyou Shen, Yunhang Zheng, Xiawu Sun, Xing Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China

The swift progress of Multi-modal Large Models (MLLMs) has showcased their impressive ability to tackle tasks blending vision and language. Yet, most current models and benchmarks cater to scenarios with a narrow scope of visual and textual contexts. These models often fall short when faced with complex comprehension tasks, which involve navigating through a plethora of irrelevant and potentially misleading information in both text and image forms. To bridge this gap, we introduce a new, more demanding task known as Interleaved Image-Text Comprehension (IITC). This task challenges models to discern and disregard superfluous elements in both images and text to accurately answer questions and to follow intricate instructions to pinpoint the relevant image. In support of this task, we further craft a new VEGA dataset, tailored for the IITC task on scientific content, and devised a subtask, Image-Text Association (ITA), to refine image-text correlation skills. Our evaluation of four leading closed-source models, as well as various open-source models using VEGA, underscores the rigorous nature of IITC. Even the most advanced models, such as Gemini-1.5-pro and GPT4V, only achieved modest success. By employing a multi-task, multi-scale post-training strategy, we have set a robust baseline for MLLMs on the IITC task, attaining an 85.8% accuracy rate in image association and a 0.508 Rouge score. These results validate the effectiveness of our dataset in improving MLLMs capabilities for nuanced image-text comprehension. Project page: https://***/VEGA/. © 2024, CC BY.

关键词： Petroleum reservoir evaluation

来源：评论

学校读者我要写书评

暂无评论

ERQ: Error Reduction for Post-Training Quantization of Vision Transformers 41

ERQ: Error Reduction for Post-Training Quantization of Visio...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Zhong, Yunshan Hu, Jiawei Huang, You Zhang, Yuxin Ji, Rongrong Institute of Artificial Intelligence Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Department of Artificial Intelligence School of Informatics Xiamen University China Peng Cheng Laboratory China

Post-training quantization (PTQ) for vision transformers (ViTs) has garnered significant attention due to its efficiency in compressing ***, existing methods typically overlook the intricate interdependence between quantized weight and activation, leading to considerable quantization *** this paper, we propose ERQ, a two-step PTQ approach meticulously crafted to sequentially reduce the quantization error arising from activation and weight *** first introduces Activation quantization error reduction (Aqer) that strategically formulates the minimization of activation quantization error as a Ridge Regression problem, tackling it by updating weights with ***, ERQ introduces Weight quantization error reduction (Wqer) that adopts an iterative approach to mitigate the quantization error induced by weight *** each iteration, an empirically derived, efficient proxy is employed to refine the rounding directions of quantized weights, coupled with a Ridge Regression solver to curtail weight quantization *** results attest to the effectiveness of our ***, ERQ surpasses the state-of-the-art GPTQ by 22.36% in accuracy for W3A4 ViT-S. Copyright 2024 by the author(s)

关键词： Error correction

来源：评论

学校读者我要写书评

暂无评论

DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysis

arXiv

引用

arXiv 2024年

作者： Chen, Zhongxi Sun, Ke Zhou, Ziyin Lin, Xianming Sun, Xiaoshuai Cao, Liujuan Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China

The rapid progress in deep learning has given rise to hyper-realistic facial forgery methods, leading to concerns related to misinformation and security risks. Existing face forgery datasets have limitations in generating high-quality facial images and addressing the challenges posed by evolving generative techniques. To combat this, we present DiffusionFace, the first diffusion-based face forgery dataset, covering various forgery categories, including unconditional and Text Guide facial image generation, Img2Img, Inpaint, and Diffusion-based facial exchange algorithms. Our DiffusionFace dataset stands out with its extensive collection of 11 diffusion models and the high-quality of the generated images, providing essential metadata and a real-world internet-sourced forgery facial image dataset for evaluation. Additionally, we provide an in-depth analysis of the data and introduce practical evaluation protocols to rigorously assess discriminative models' effectiveness in detecting counterfeit facial images, aiming to enhance security in facial image authentication processes. The dataset is available for download at https://***/Rapisurazurite/DiffFace. Copyright © 2024, The Authors. All rights reserved.

关键词： Diffusion

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：