检索结果-内蒙古大学图书馆

Towards high-throughput and low-latency billion-scale vector search via CPU/GPU collaborative filtering and re-ranking 25

学校读者我要写书评

暂无评论

Towards high-throughput and low-latency billion-scale vector...

Proceedings of the 23rd USENIX Conference on File and Storage Technologies

作者： Bing Tian Haikun Liu Yuhang Tang Shihai Xiao Zhuohui Duan Xiaofei Liao Hai Jin Xuecang Zhang Junhua Zhu Yu Zhang National Engineering Research Center for Big Data Technology and System Service Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China Huawei Technologies Co. Ltd Towards high-throughput and low-latency billion-scale vector search via CPU/GPU collaborative filtering and re-ranking

ISBN: (纸本)9781939133458

Approximate nearest neighbor search (ANNS) has emerged as a crucial component of database and AI infrastructure. Ever-increasing vector datasets pose significant challenges in terms of performance, cost, and accuracy for ANNS services. None of modern ANNS systems can address these issues simultaneously. In this paper, we present Fusion-ANNS, a high-throughput, low-latency, cost-efficient, and high-accuracy ANNS system for billion-scale datasets using SSDs and only one entry-level GPU. The key idea of Fusion-ANNS lies in CPU/GPU collaborative filtering and reranking mechanisms, which significantly reduce I/O operations across CPUs, GPU, and SSDs to break through the I/O performance bottleneck. Specifically, we propose three novel designs: (1) multi-tiered indexing to avoid data swapping between CPUs and GPU, (2) heuristic re-ranking to eliminate unnecessary I/Os and computations while guaranteeing high accuracy, and (3) redundant-aware I/O deduplication to further improve I/O efficiency. We implement FusionANNS and compare it with the state-of-the-art SSD-based ANNS system-SPANN and GPU-accelerated in-memory ANNS system-RUMMY. Experimental results show that FusionANNS achieves 1) 9.4-13.1× higher query per second (QPS) and 5.7-8.8× higher cost efficiency compared with SPANN; 2) and 2-4.9× higher QPS and 2.3-6.8× higher cost efficiency compared with RUMMY, while guaranteeing low latency and high accuracy.

关键词：

Automated data Visualization from Natural Language via Large Language Models: An Exploratory Study

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Yang Wan, Yao Zhang, Hongyu Sui, Yulei Wei, Wucai Zhao, Wei Xu, Guandong Jin, Hai Huazhong University of Science and Technology China Chongqing University China University of New South Wales Australia University of Technology Sydney Australia National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

The Natural Language to Visualization (NL2Vis) task aims to transform natural-language descriptions into visual representations for a grounded table, enabling users to gain insights from vast amounts of data. Recently, many deep learning-based approaches have been developed for NL2Vis. Despite the considerable efforts made by these approaches, challenges persist in visualizing data sourced from unseen databases or spanning multiple tables. Taking inspiration from the remarkable generation capabilities of Large Language Models (LLMs), this paper conducts an empirical study to evaluate their potential in generating visualizations, and explore the effectiveness of in-context learning prompts for enhancing this task. In particular, we first explore the ways of transforming structured tabular data into sequential text prompts, as to feed them into LLMs and analyze which table content contributes most to the NL2Vis. Our findings suggest that transforming structured tabular data into programs is effective, and it is essential to consider the table schema when formulating prompts. Furthermore, we evaluate two types of LLMs: finetuned models (e.g., T5-Small) and inference-only models (e.g., GPT-3.5), against state-of-the-art methods, using the NL2Vis benchmarks (i.e., nvBench). The experimental results reveal that LLMs outperform baselines, with inference-only models consistently exhibiting performance improvements, at times even surpassing fine-tuned models when provided with certain few-shot demonstrations through in-context learning. Finally, we analyze when the LLMs fail in NL2Vis, and propose to iteratively update the results using strategies such as chain-of-thought, role-playing, and code-interpreter. The experimental results confirm the efficacy of iterative updates and hold great potential for future study. © 2024, CC BY.

关键词： data visualization

Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalization

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yuan, Haonan Sun, Qingyun Fu, Xingcheng Zhang, Ziwei Ji, Cheng Peng, Hao Li, Jianxin Beijing Advanced Innovation Center for Big Data and Brain Computing Beihang University China School of Computer Science and Engineering Beihang University China Key Lab of Education Blockchain and Intelligent Technology Guangxi Normal University China Department of Computer Science and Technology Tsinghua University China

Dynamic graph neural networks (DGNNs) are increasingly pervasive in exploiting spatio-temporal patterns on dynamic graphs. However, existing works fail to generalize under distribution shifts, which are common in real-world scenarios. As the generation of dynamic graphs is heavily influenced by latent environments, investigating their impacts on the out-of-distribution (OOD) generalization is critical. However, it remains unexplored with the following two major challenges: (1) How to properly model and infer the complex environments on dynamic graphs with distribution shifts? (2) How to discover invariant patterns given inferred spatio-temporal environments? To solve these challenges, we propose a novel Environment-Aware dynamic Graph LEarning (EAGLE) framework for OOD generalization by modeling complex coupled environments and exploiting spatiotemporal invariant patterns. Specifically, we first design the environment-aware EA-DGNN to model environments by multi-channel environments disentangling. Then, we propose an environment instantiation mechanism for environment diversification with inferred distributions. Finally, we discriminate spatio-temporal invariant patterns for out-of-distribution prediction by the invariant pattern recognition mechanism and perform fine-grained causal interventions node-wisely with a mixture of instantiated environment samples. Experiments on real-world and synthetic dynamic graph datasets demonstrate the superiority of our method against state-of-the-art baselines under distribution shifts. To the best of our knowledge, we are the first to study OOD generalization on dynamic graphs from the environment learning perspective. Copyright © 2023, The Authors. All rights reserved.

关键词： Graph neural networks

Online Q&A for Machine Learning Courses Based on Formula Recognition

学校读者我要写书评

暂无评论

Online Q&A for Machine Learning Courses Based on Formula Rec...

Information Science and Education (ICISE-IE), International Conference on

作者： Xiaoke Zhu Xiaopan Chen Minghao Li Tianxiang Zhang School of Computer and Information Engineering Henan University Kaifeng China Henan Engineering Research Center of Intelligent Technology and Application Henan University Kaifeng China Henan Key Laboratory of Big Data Analysis and Processing Henan University Kaifeng China

Mathematical formulas are ubiquitous in courses such as Machine Learning. Understanding these formulas is the primary challenge for studying of students. If a searchable knowledge base can be built for these formulas, it will help students learn. Accurate identification of these formulas is an important step in building and querying the knowledge base. However, mathematical formulas have complex two-dimensional and nested structures, which makes them inconvenient to search like ordinary text. This paper employs Faster R-CNN to detect regions containing mathematical formulas and improves its backbone network, anchor box dimensions, and pre-training network to make it more accurate and efficient in identifing these formulas. Furthermore, in order to enhance the features and meaningful representation of mathematical formulas in the input image, we introduce the distance transformation. Finally, comparative experiments were conducted on the publicly dataset GTDB. Our proposed approach achieved recognition rates of 91.22% and 85.74% for isolated and hybrid formulas detection.

关键词：

Medical image super-resolution algorithm based on multi-scale feature aggregation

学校读者我要写书评

暂无评论

Medical image super-resolution algorithm based on multi-scal...

Medical Artificial Intelligence (MedAI), IEEE International Conference on

作者： Xinyu Han Xun Gong School of Computing and Artificial Intelligence Southwest Jiaotong University Chengdu China Engineering Research Center of Sustainable Urban Intelligent Transportation Ministry of Education Chengdu China National Engineering Laboratory of Integrated Transportation Big Data Application Technology Southwest Jiaotong University Chengdu China Manufacturing Industry Chains Collaboration and InformationSupport Technology Key Laboratory of Sichuan Province Southwest Jiaotong University Chengdu China

ISBN: (数字)9798350377613

ISBN: (纸本)9798350377620

Ultrasound images are vital for medical diagnostics but often suffer from information loss and blurred details due to limitations in imaging systems and sensor technologies. Many researchers have proposed super-resolution algorithms to enhance medical images. But existing super-resolution methods struggle with the uneven distribution of noise and clarity. To address this, we propose a super-resolution algorithm for medical images based on multi-scale feature aggregation Leveraging the architecture of Unet as our primary framework, our method enhances output details through multi-scale hole convolutions. Taking into consideration the characteristics of ultrasonic images, we propose a frequency domain-based module to enhances edge information while effectively denoising the image. Furthermore, to improve the quality of the output images, we introduce an image processing module grounded in global information. The module ensures clarity while considering the overarching context, thereby preserving global consistency and enhancing output quality. We experiment on three datasets to demonstrate the effectiveness of our model. Additionally, significant improvements in medical image segmentation are observed, proving the practicability of our proposed approach.

关键词： Image segmentation Ultrasonic imaging Image edge detection Frequency-domain analysis Superresolution Noise reduction Noise Medical diagnosis Artificial intelligence Medical diagnostic imaging

Enhancing Online Teaching Effectiveness Through Hand-Raising Detection

学校读者我要写书评

暂无评论

Enhancing Online Teaching Effectiveness Through Hand-Raising...

Information Science and Education (ICISE-IE), International Conference on

作者： Xiaoke Zhu Zhiwei Dong Xiaopan Chen Yufeng Shi Henan Engineering Research Center of Intelligent Technology and Application Henan University Kaifeng China School of Computer and Information Engineering Henan University Kaifeng China Henan Key Laboratory of Big Data Analysis and Processing Henan University Kaifeng China

After the outbreak of the COVID-19 pandemic, online teaching has gradually emerged as an indispensable component of education. Despite its convenience, online education lacks the immediate interactive experience inherent in traditional classrooms. This paper presents a meticulously designed DS-YOLOv8 network model, specifically tailored for real-time recognition of students raising their hands under cameras. On our proprietary dataset, this network model exhibits exceptional accuracy in hand-raising detection, showcasing a 1.3% improvement in the AP50 index compared to the baseline YOLOv8 model. Our focus lies on the application of hand-raising detection within online classrooms with an objective to provide enhanced support for teachers by promptly identifying students raising their hands and thereby improving teaching efficiency while enriching the interactive experience of students in virtual learning environments.

关键词：

Improving Bert Fine-Tuning via Stabilizing Cross-Layer Mutual Information

学校读者我要写书评

暂无评论

Improving Bert Fine-Tuning via Stabilizing Cross-Layer Mutua...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Jicun Li Xingjian Li Tianyang Wang Shi Wang Yanan Cao Chengzhong Xu Dejing Dou Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences Big Data Lab Baidu Research State Key Lab of IOTSC University of Macau University of Alabama at Birmingham Institute of Information Engineering Chinese Academy of Sciences

Fine-tuning pre-trained language models, such as BERT, has shown enormous success among various NLP tasks. Though simple and effective, the process of fine-tuning has been found unstable, which often leads to unexpected poor performance. To increase stability and generalizability, most existing works resort to maintaining the parameters or representations of pre-trained models during fine-tuning. Nevertheless, very little work explores mining the reliable part of pre-learned information that can help to stabilize fine-tuning. To address this challenge, we introduce a novel solution in which we fine-tune BERT with stabilized cross-layer mutual information. Our method aims to preserve the reliable behaviors of cross-layer information propagation, instead of preserving the information itself, of the pre-trained model. Therefore, our method circumvents the domain conflicts between pre-trained and target tasks. We conduct extensive experiments with popular pre-trained BERT variants on NLP datasets, demonstrating the universal effectiveness and robustness of our method.

关键词： Cross layer design Bit error rate Signal processing Stability analysis Robustness Acoustics Behavioral sciences

OCDB: REVISITING CAUSAL DISCOVERY WITH A COMPREHENSIVE BENCHMARK AND EVALUATION FRAMEWORK

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Wei Huang, Hong Zhang, Guowen Shi, Ruize Yin, Kehan Lin, Yuanyuan Liu, Bang Huazhong University of Science and Technology China DIRO Université de Montréal & Mila Canada CIFAR AI Chair Canada The National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

Large language models (LLMs) have excelled in various natural language processing tasks, but challenges in interpretability and trustworthiness persist, limiting their use in high-stakes fields. Causal discovery offers a promising approach to improve transparency and reliability. However, current evaluations are often one-sided and lack assessments focused on interpretability performance. Additionally, these evaluations rely on synthetic data and lack comprehensive assessments of real-world datasets. These lead to promising methods potentially being overlooked. To address these issues, we propose a flexible evaluation framework with metrics for evaluating differences in causal structures and causal effects, which are crucial attributes that help improve the interpretability of LLMs. We introduce the Open Causal Discovery Benchmark (OCDB), based on real data, to promote fair comparisons and drive optimization of algorithms. Additionally, our new metrics account for undirected edges, enabling fair comparisons between Directed Acyclic Graphs (DAGs) and Completed Partially Directed Acyclic Graphs (CPDAGs). Experimental results show significant shortcomings in existing algorithms' generalization capabilities on real data, highlighting the potential for performance improvement and the importance of our framework in advancing causal discovery techniques. Copyright © 2024, The Authors. All rights reserved.

关键词： Directed graphs

Shielding Federated Learning: Robust Aggregation with Adaptive Client Selection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wan, Wei Hu, Shengshan Lu, Jianrong Yu Zhang, Leo Jin, Hai He, Yuanyuan School of Cyber Science and Engineering Huazhong University of Science and Technology China School of Computer Science and Technology Huazhong University of Science and Technology China National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Hubei Engineering Research Center on Big Data Security China Cluster and Grid Computing Lab School of Information Technology Deakin University Australia

Federated learning (FL) enables multiple clients to collaboratively train an accurate global model while protecting clients’ data privacy. However, FL is susceptible to Byzantine attacks from malicious participants. Although the problem has gained significant attention, existing defenses have several flaws: the server irrationally chooses malicious clients for aggregation even after they have been detected in previous rounds;the defenses perform ineffectively against sybil attacks or in the heterogeneous data setting. To overcome these issues, we propose MAB-RFL, a new method for robust aggregation in FL. By modelling the client selection as an extended multi-armed bandit (MAB) problem, we propose an adaptive client selection strategy to choose honest clients that are more likely to contribute high-quality updates. We then propose two approaches to identify malicious updates from sybil and non-sybil attacks, based on which rewards for each client selection decision can be accurately evaluated to discourage malicious behaviors. MAB-RFL achieves a satisfying balance between exploration and exploitation on the potential benign clients. Extensive experimental results show that MAB-RFL outperforms existing defenses in three attack scenarios under different percentages of attackers. Copyright © 2022, The Authors. All rights reserved.

关键词： data privacy