检索结果-内蒙古大学图书馆

Statistical knowledge assessment for large language models 23

学校读者我要写书评

暂无评论

Statistical knowledge assessment for large language models

Proceedings of the 37th International Conference on Neural information processing Systems

作者： Qingxiu Dong Jingjing Xu Lingpeng Kong Zhifang Sui Lei Li National Key Laboratory for Multimedia Information Processing School of Computer Science Peking University Shanghai AI Lab The University of Hong Kong Carnegie Mellon University

Given varying prompts regarding a factoid question, can a large language model (LLM) reliably generate factually correct answers? Existing LLMs may generate distinct responses for different prompts. In this paper, we study the problem of quantifying knowledge contained in an LLM regarding a given set of facts. We propose KaRR, a statistical approach to assess factual knowledge for LLMs. The main idea is to estimate the ratio of LLM generating text corresponding to the answer entity given diverse prompts of the subject and the querying relation, versus it generating by random chances. Our assessment suite contains a comprehensive set of 994,123 entities and 600 relations, with 1,395,905 text aliases. We use our method to evaluate 20 LLMs of various sizes, including LLaMA, Alpaca, OPT, etc. Experiments show that our results have a strong correlation (0.43 Kendall's τ) with the results of human assessment on LLMs. Our results reveal that the knowledge in LLMs with the same backbone architecture adheres to the scaling law, while tuning on instruction-following data sometimes compromises the model's capability to generate factually correct text reliably.

关键词：

Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functions

学校读者我要写书评

暂无评论

Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Yun He Danhang Tang Yinda Zhang Xiangyang Xue Yanwei Fu School of Computer Science Fudan University. Google School of Data Science Fudan University Shanghai Key Lab of Intelligent Information Processing Fudan ISTBI�ZJNU Algorithm Centre for Brain-inspired Intelligence Zhejiang Normal University Jinhua China

Most existing point cloud upsampling methods have roughly three steps: feature extraction, feature expansion and 3D coordinate prediction. However, they usually suffer from two critical issues: (1) fixed upsampling rate after one-time training, since the feature expansion unit is customized for each upsampling rate; (2) outliers or shrinkage artifact caused by the difficulty of precisely predicting 3D coordinates or residuals of upsampled points. To adress them, we propose a new framework for accurate point cloud upsampling that supports arbitrary upsampling rates. Our method first interpolates the low-res point cloud according to a given upsampling rate. And then refine the positions of the interpolated points with an iterative optimization process, guided by a trained model estimating the difference between the current point cloud and the high-res target. Extensive quantitative and qualitative results on benchmarks and downstream tasks demonstrate that our method achieves the state-of-the-art accuracy and efficiency.

关键词：

PG-Attack: A Precision-Guided Adversarial Attack Framework Against Vision Foundation Models for Autonomous Driving

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fu, Jiyuan Chen, Zhaoyu Jiang, Kaixun Guo, Haijing Gao, Shuyong Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University China

Vision foundation models are increasingly employed in autonomous driving systems due to their advanced capabilities. However, these models are susceptible to adversarial attacks, posing significant risks to the reliability and safety of autonomous vehicles. Adversaries can exploit these vulnerabilities to manipulate the vehicle’s perception of its surroundings, leading to erroneous decisions and potentially catastrophic consequences. To address this challenge, we propose a novel Precision-Guided Adversarial Attack (PG-Attack) framework that combines two techniques: Precision Mask Perturbation Attack (PMP-Attack) and Deceptive Text Patch Attack (DTP-Attack). PMP-Attack precisely targets the attack region to minimize the overall perturbation while maximizing its impact on the target object’s representation in the model’s feature space. DTP-Attack introduces deceptive text patches that disrupt the model’s understanding of the scene, further enhancing the attack’s effectiveness. Our experiments demonstrate that PG-Attack successfully deceives a variety of advanced multi-modal large models, including GPT-4V, Qwen-VL, and imp-V1. Additionally, we won First-Place in the CVPR 2024 Workshop Challenge: Black-box Adversarial Attacks on Vision Foundation Models‡ and codes are available at https://***/fuhaha824/PG-Attack. Copyright © 2024, The Authors. All rights reserved.

关键词： Autonomous vehicles

Trace Monomial Boolean Functions with Large High-Order Nonlinearities

学校读者我要写书评

暂无评论

arXiv 2023年

Exhibiting an explicit Boolean function with a large high-order nonlinearity is an important problem in cryptography, coding theory, and computational complexity. We prove lower bounds on the second-order, third-order, and higher-order nonlinearities of some trace monomial Boolean functions. We prove lower bounds on the second-order nonlinearities of functions trn(x7) and trn(x2r+3) where n = 2r. Among all trace monomials, our bounds match the best second-order nonlinearity lower bounds by [Car08] and [YT20] for odd and even n respectively. We prove a lower bound on the third-order nonlinearity for functions trn(x15), which is the best third-order nonlinearity lower bound. For any r, we prove that the r-th order nonlinearity of trn(x2r+1−1) is at least (Formula Presented). For r log2 n, this is the best lower bound among all explicit functions. © 2023, CC BY.

关键词： Boolean functions

A Survey on RGB, 3D, and Multimodal Approaches for Unsupervised Industrial Image Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Yuxuan Chang, Yang Tong, Xuan Yu, Jiawen Liotta, Antonio Huang, Guofan Song, Wei Zeng, Deyu Wu, Zongze Wang, Yan Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai200438 China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai200438 China Faculty of Computer Science Free University of Bozen-Bolzano Bolzano39100 Italy College of Information Technology Shanghai Ocean University Shanghai201306 China College of Mechatronics and Control Engineering Shenzhen University Shenzhen518060 China

In the advancement of industrial informatization, unsupervised anomaly detection technology effectively overcomes the scarcity of abnormal samples and significantly enhances the automation and reliability of smart manufacturing. As an important branch, industrial image anomaly detection focuses on automatically identifying visual anomalies in industrial scenarios (such as product surface defects, assembly errors, and equipment appearance anomalies) through computer vision techniques. With the rapid development of Unsupervised industrial Image Anomaly Detection (UIAD), excellent detection performance has been achieved not only in RGB setting but also in 3D and multimodal (RGB and 3D) settings. However, existing surveys primarily focus on UIAD tasks in RGB setting, with little discussion in 3D and multimodal settings. To address this gap, this artical provides a comprehensive review of UIAD tasks in the three modal settings. Specifically, we first introduce the task concept and process of UIAD. We then overview the research on UIAD in three modal settings (RGB, 3D, and multimodal), including datasets and methods, and review multimodal feature fusion strategies in multimodal setting. Finally, we summarize the main challenges faced by UIAD tasks in the three modal settings, and offer insights into future development directions, aiming to provide researchers with a comprehensive reference and offer new perspectives for the advancement of industrial informatization. Corresponding resources are available at https://***/Sunny5250/Awesome-Multi-Setting-UIAD. Copyright © 2024, The Authors. All rights reserved.

关键词： Smart manufacturing

Improving Adversarial Transferability with Neighbourhood Gradient information

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guo, Haijing Wang, Jiafeng Chen, Zhaoyu Jiang, Kaixun Hong, Lingyi Guo, Pinxue Li, Jinglun Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai200433 China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai200433 China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai200433 China

Deep neural networks (DNNs) are known to be susceptible to adversarial examples, leading to significant performance degradation. In black-box attack scenarios, a considerable attack performance gap between the surrogate model and the target model persists. This work focuses on enhancing the transferability of adversarial examples to narrow this performance gap. We observe that the gradient information around the clean image, i.e. Neighbourhood Gradient information, can offer high transferability. Leveraging this, we propose the NGI-Attack, which incorporates Example Backtracking and Multiplex Mask strategies, to use this gradient information and enhance transferability fully. Specifically, we first adopt Example Backtracking to accumulate Neighbourhood Gradient information as the initial momentum term. Multiplex Mask, which forms a multi-way attack strategy, aims to force the network to focus on non-discriminative regions, which can obtain richer gradient information during only a few iterations. Extensive experiments demonstrate that our approach significantly enhances the adversarial transferability. Especially, when attacking numerous defense models, we achieve an average attack success rate of 95.8%. Notably, our method can plugin with any off-the-shelf algorithm to improve their attack performance without additional time cost. Copyright © 2024, The Authors. All rights reserved.

关键词：

Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Junxiong Wang, Yan Tao, Zeng Wang, Boyang Zhao, Qing Wang, Haorang Tong, Xuan Mai, Xinji Lin, Yuxuan Song, Wei Yu, Jiawen Yan, Shaoqi Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China East China University of Science and Technology Shanghai China College of Information Technology Shanghai Ocean University Shanghai China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

Pre-trained diffusion models utilized for image generation encapsulate a substantial reservoir of a priori knowledge pertaining to intricate textures. Harnessing the potential of leveraging this a priori knowledge in the context of image super-resolution presents a compelling avenue. Nonetheless, prevailing diffusion-based methodologies presently overlook the constraints imposed by degradation information on the diffusion process. Furthermore, these methods fail to consider the spatial variability inherent in the estimated blur kernel, stemming from factors such as motion jitter and out-of-focus elements in open-environment scenarios. This oversight results in a notable deviation of the image super-resolution effect from fundamental realities. To address these concerns, we introduce a framework known as Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution (SSR). Within the SSR framework, we propose a Spatially Variant Kernel Refinement (SVKR) module. SVKR estimates a Depth-Informed Kernel, which takes the depth information into account and is spatially variant. Additionally, SVKR enhance the accuracy of depth information acquired from LR images, allowing for mutual enhancement between the depth map and blur kernel estimates. Finally, we introduce the Adaptive Multi-Modal Fusion (AMF) module to align the information from three modalities: low-resolution images, depth maps, and blur kernels. This alignment can constrain the diffusion model to generate more authentic SR results. Copyright © 2024, The Authors. All rights reserved.

关键词： Image enhancement

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ye, Junjie Li, Guanyu Gao, Songyang Huang, Caishuang Wu, Yilong Li, Sixian Fan, Xiaoran Dou, Shihan Ji, Tao Zhang, Qi Gui, Tao Huang, Xuanjing School of Computer Science Fudan University China Institute of Modern Languages and Linguistics Fudan University China Key Laboratory of Intelligent Information Processing Fudan University Shanghai China Research Institute of Intelligent Complex Systems Fudan University China Pengcheng Laboratory China

Existing evaluations of tool learning primarily focus on validating the alignment of selected tools (e.g., various APIs) for large language models (LLMs) with expected outcomes. However, these approaches rely on a limited set of scenarios where answers can be pre-determined. Furthermore, a sole emphasis on outcomes disregards the complex capabilities required for LLMs to effectively use tools. To tackle this issue, we propose ToolEyes, a fine-grained system tailored for the evaluation of the LLMs’ tool learning capabilities in authentic scenarios. The system meticulously examines seven real-world scenarios, analyzing five dimensions crucial to LLMs in tool learning: format alignment, intent comprehension, behavior planning, tool selection, and answer organization. Additionally, ToolEyes incorporates a tool library boasting approximately 600 tools, serving as an intermediary between LLMs and the physical world. Evaluations involving ten LLMs across three categories reveal a preference for specific scenarios and limited cognitive abilities in tool learning. Intriguingly, expanding the model size even exacerbates the hindrance to tool learning. The code and data are available at https://***/Junjie-Ye/ToolEyes. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity

学校读者我要写书评

暂无评论

UFineBench: Towards Text-based Person Retrieval with Ultra-f...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Jialong Zuo Hanyu Zhou Ying Nie Feng Zhang Tianyu Guo Nong Sang Yunhe Wang Changxin Gao National Key Laboratory of Multispectral Information Intelligent Processing Technology School of Artificial Intelligence and Automation Huazhong University of Science and Technology Huawei Noah's Ark Lab

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Existing text-based person retrieval datasets often have relatively coarse-grained text annotations. This hinders the model to comprehend the fine-grained semantics of query texts in real scenarios. To address this problem, we con-tribute a new benchmark named UFineBench for text-based person retrieval with ultra-fine granularity. Firstly, we construct a new dataset named UFine6926. We collect a large number of person images and manually annotate each image with two detailed textual descriptions, averaging 80.8 words each. The average word count is three to four times that of the previous datasets. In addition of standard in-domain evaluation, we also propose a spe-cial evaluation paradigm more representative of real sce-narios. It contains a new evaluation set with cross domains, cross textual granularity and cross textual styles, named UFine3C, and a new evaluation metric for accurately mea-suring retrieval ability, named mean Similarity Distribution (mSD). Moreover, we propose CFAM, a more efficient al-gorithm especially designed for text-based person retrieval with ultra fine-grained texts. It achieves fine granularity mining by adopting a shared cross-modal granularity de-coder and hard negative match mechanism. With standard in-domain evaluation, CFAM establishes competitive performance across various datasets, espe-cially on our ultra fine-grained UFine6926. Furthermore, by evaluating on UFine3C, we demonstrate that training on our UFine6926 significantly improves generalization to real scenarios compared with other coarse-grained datasets. The dataset and code will be made publicly available at https://***/Zplusdragon/UFineBench.

关键词： Training Measurement computer vision Correlation Codes Annotations Semantics