检索结果-内蒙古大学图书馆

Multi-Grained Multimodal interaction Network for Entity Linking

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Luo, Pengfei Xu, Tong Wu, Shiwei Zhu, Chen Xu, Linli Chen, Enhong School of Computer Science and Technology University of Science and Technology of China State Key Laboratory of Cognitive Intelligence Anhui Hefei China School of Data Science University of Science and Technology of China State Key Laboratory of Cognitive Intelligence Anhui Hefei China Career Science Lab BOSS Zhipin School of Management University of Science and Technology of China Beijing China

Multimodal entity linking (MEL) task, which aims at resolving ambiguous mentions to a multimodal knowledge graph, has attracted wide attention in recent years. Though large efforts have been made to explore the complementary effect among multiple modalities, however, they may fail to fully absorb the comprehensive expression of abbreviated textual context and implicit visual indication. Even worse, the inevitable noisy data may cause inconsistency of different modalities during the learning process, which severely degenerates the performance. To address the above issues, in this paper, we propose a novel Multi-GraIned Multimodal interaction Network (MIMIC) framework for solving the MEL task. Specifically, the unified inputs of mentions and entities are first encoded by textual/visual encoders separately, to extract global descriptive features and local detailed features. Then, to derive the similarity matching score for each mention-entity pair, we device three interaction units to comprehensively explore the intra-modal interaction and inter-modal fusion among features of entities and mentions. In particular, three modules, namely the Text-based Global-Local interaction Unit (TGLU), Vision-based DuaL interaction Unit (VDLU) and Cross-Modal Fusion-based interaction Unit (CMFU) are designed to capture and integrate the fine-grained representation lying in abbreviated text and implicit visual cues. Afterwards, we introduce a unit-consistency objective function via contrastive learning to avoid inconsistency and model degradation. Experimental results on three public benchmark datasets demonstrate that our solution outperforms various state-of-the-art baselines, and ablation studies verify the effectiveness of designed modules. © 2023, CC BY-NC-ND.

关键词： Learning systems

Confidence-oriented Contrastive Graph Clustering

学校读者我要写书评

暂无评论

Confidence-oriented Contrastive Graph Clustering

International Joint Conference on Neural Networks (IJCNN)

作者： Yan-Di Huang Guang-Yu Zhang Dong Huang Chang-Dong Wang Yang Liu Enbo Huang College of Mathematics and Informatics South China Agricultural University Guangzhou China School of Computer Science and Engineering Sun Yat-Sen University Guangdong Provincial Key Laboratory of Intellectual Property and Big Data Guangzhou China School of Computer Science and Engineering Sun Yat-Sen University Guangzhou China Guangxi Key Lab of Human-Machine Interaction and Intelligent Decision Nanning Normal University Nanning China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Contrastive clustering has recently been an emerging topic in deep unsupervised learning. Nevertheless, the previous works mostly adopt the stochastic data augmentations, which easily leads to the semantic drift problem by limited transformations. Moreover, these approaches ignore the data distribution information when generating the positive and negative pair-wise samples. In light of this, this paper proposes a simple yet effective unsupervised clustering network termed Confidence-oriented Contrastive Graph Clustering (CoCGC). Particularly, we design an end-to-end network paradigm with un-shared weights, among which a hybrid graph filter is utilized to generate two views of reliable augmentations. Guided by the non-dominated sorting theory, we further construct a confidence-oriented sample set from the latent data distribution perspective. By considering the local density and cluster distribution of the embedding representations, the discriminative sample pairs can be derived from the confidence-oriented sets in a two-view contrastive manner. Finally, a cross-view neighbor contrastive loss is devised for better exploiting the self-supervised network signals. Extensive experimental results on five benchmark datasets demonstrate the effectiveness of our method against the existing state-of-the-art deep graph clustering methods.

关键词： Network topology Semantics Neural networks Reliability theory Reliability engineering Information filters Filtering theory

A Comprehensive Survey of Action Quality Assessment: Method and Benchmark

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Kanglei Cai, Ruizhi Wang, Liyuan Shum, Hubert P.H. Liang, Xiaohui State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China The Department of Computer Science and Technology Institute for AI BNRist Center THBI Lab Tsinghua-Bosch Joint Center for ML Tsinghua University Beijing100190 China The Department of Computer Science Durham University DurhamDH1 3LE United Kingdom The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China Zhongguancun Laboratory Beijing China

Action Quality Assessment (AQA) quantitatively evaluates the quality of human actions, providing automated assessments that reduce biases in human judgment. Its applications span domains such as sports analysis, skill assessment, and medical care. Recent advances in AQA have introduced innovative methodologies, but similar methods often intertwine across different domains, highlighting the fragmented nature that hinders systematic reviews. In addition, the lack of a unified benchmark and limited computational comparisons hinder consistent evaluation and fair assessment of AQA approaches. In this work, we address these gaps by systematically analyzing over 150 AQA-related papers to develop a hierarchical taxonomy, construct a unified benchmark, and provide an in-depth analysis of current trends, challenges, and future directions. Our hierarchical taxonomy categorizes AQA methods based on input modalities (video, skeleton, multi-modal) and their specific characteristics, highlighting the evolution and interrelations across various approaches. To promote standardization, we present a unified benchmark, integrating diverse datasets to evaluate the assessment precision and computational efficiency. Finally, we review emerging task-specific applications and identify under-explored challenges in AQA, providing actionable insights into future research directions. This survey aims to deepen understanding of AQA progress, facilitate method comparison, and guide future innovations. The project web page can be found at https://***/AQA-Survey. Copyright © 2024, The Authors. All rights reserved.

关键词：

Enhanced Fine-Tuning of Lightweight Domain-Specific Q&A Model Based on Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Shenglin Zhu, Pengtian Ma, Minghua Wang, Jiagang Sun, Yongqian Li, Dongwen Wang, Jingyu Guo, Qianying Hua, Xiaolei Zhu, Lin Pei, Dan Nankai University China Microsoft United States Tsinghua University China China Mobile Research Institute China Haihe Laboratory of Information Technology Application Innovation China Tianjin Key Laboratory of Software Experience and Human Computer Interaction China Beijing National Research Center for Information Science and Technology China

Large language models (LLMs) excel at general question-answering (Q&A) but often fall short in specialized domains due to a lack of domain-specific knowledge. Commercial companies face the dual challenges of privacy protection and resource constraints when involving LLMs for fine-tuning. This paper propose a novel framework, Self-Evolution, designed to address these issues by leveraging lightweight open-source LLMs through multiple iterative fine-tuning rounds. To enhance the efficiency of iterative fine-tuning, Self-Evolution employ a strategy that filters and reinforces the knowledge with higher value during the iterative process. We employed Self-Evolution on Qwen1.5-7B-Chat using 4,000 documents containing rich domain knowledge from China Mobile, achieving a performance score 174% higher on domain-specific question-answering evaluations than Qwen1.5-7B-Chat and even 22% higher than Qwen1.5-72B-Chat. Self-Evolution has been deployed in China Mobile's daily operation and maintenance for 117 days, and it improves the efficiency of locating alarms, fixing problems, and finding related reports, with an average efficiency improvement of over 18.6%. In addition, we release Self-Evolution framework code in https://***/Zero-Pointer/Self-Evolution. Copyright © 2024, The Authors. All rights reserved.

关键词： Data mining

Learning to predict explainable plots for neural story generation

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Chen, Gang Liu, Yang Luan, Huanbo Zhang, Meng Liu, Qun Sun, Maosong Institute for Artificial Intelligence State Key Laboratory of Intelligent Technology and Systems Department of Computer Science and Technology Tsinghua University Beijing China Beijing National Research Center for Information Science and Technology Noah's Ark Lab Paris Huawei Technologies Ltd

Story generation is an important natural language processing task that aims to generate coherent stories automatically. While the use of neural networks has proven effective in improving story generation, how to learn to generate an explainable high-level plot still remains a major challenge. In this work, we propose a latent variable model for neural story generation. The model treats an outline, which is a natural language sentence explainable to humans, as a latent variable to represent a high-level plot that bridges the input and output. We adopt an external summarization model to guide the latent variable model to learn how to generate outlines from training data. Experiments show that our approach achieves significant improvements over state-of-the-art methods in both automatic and human evaluations. Copyright © 2019, The Authors. All rights reserved.

关键词： Natural language processing systems

Rethinking natural adversarial examples for classification models

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Xiao Li, Jianmin Dai, Ting Shi, Jie Zhu, Jun Hu, Xiaolin Beijing National Research Center for Information Science and Technology State Key Laboratory of Intelligent Technology and Systems Tsinghua University Beijing100084 China Department of Computer Science and Technology Tsinghua University Beijing100084 China Trustworthy AI Lab of Shield Lab at Huawei Singapore Research Center Singapore

Recently, it was found that many real-world examples without intentional modifications can fool machine learning models, and such examples are called "natural adversarial examples". ImageNet-A is a famous dataset of natural adversarial examples. By analyzing this dataset, we hypothesized that large, cluttered and/or unusual background is an important reason why the images in this dataset are difficult to be classified. We validated the hypothesis by reducing the background influence in ImageNet-A examples with object detection techniques. Experiments showed that the object detection models with various classification models as backbones obtained much higher accuracy than their corresponding classification models. A detection model based on the classification model EfficientNet-B7 achieved a top-1 accuracy of 53.95%, surpassing previous state-of-the-art classification models trained on ImageNet, suggesting that accurate localization information can significantly boost the performance of classification models on ImageNet-A. We then manually cropped the objects in images from ImageNet-A and created a new dataset, named ImageNet-A-Plus. A human test on the new dataset showed that the deep learning-based classifiers still performed quite poorly compared with humans. Therefore, the new dataset can be used to study the robustness of classification models to the internal variance of objects without considering the background disturbance. Copyright © 2021, The Authors. All rights reserved.

关键词： Object recognition

Deep ranking: Triplet MatchNet for music metric learning

学校读者我要写书评

暂无评论

Deep ranking: Triplet MatchNet for music metric learning

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Rui Lu Kailun Wu Zhiyao Duan Changshui Zhang Department of Automation Tsinghua University State Key Lab of Intelligent Technologies and Systems Tsinghua National Laboratory for Information Science and Technology (TNList) Beijing China Department of Electrical and Computer Engineering University of Rochester NY USA

ISBN: (纸本)9781509041183

Metric learning for music is an important problem for many music information retrieval (MIR) applications such as music generation, analysis, retrieval, classification and recommendation. Traditional music metrics are mostly defined on linear transformations of handcrafted audio features, and may be improper in many situations given the large variety of music styles and instrumentations. In this paper, we propose a deep neural network named Triplet MatchNet to learn metrics directly from raw audio signals of triplets of music excerpts with human-annotated relative similarity in a supervised fashion. It has the advantage of learning highly nonlinear feature representations and metrics in this end-to-end architecture. Experiments on a widely used music similarity measure dataset show that our method significantly outperforms three state-of-the-art music metric learning methods. Experiments also show that the learned features better preserve the partial orders of the relative similarity than handcrafted features.

关键词： Metric learning music similarity deep learning convolutional neural networks

P-31: Visual Fatigue Assessment and Modeling Based on ECG and EOG Caused by 2D and 3D Displays

学校读者我要写书评

暂无评论

SID Symposium Digest of Technical Papers 2016年第1期47卷

作者： Xinpan Yang Danli Wang Haichen Hu Kang Yue Beijing Key Lab of Human-Computer Interaction Institute of Software Chinese Academy of Sciences Beijing China State Key Laboratory of Computer Institute of Software Chinese Academy of Sciences Beijing China

Three-dimensional (3D) displays become more and more popular in many fields, because they can provide amazing visual effects. However, visual fatigue as one of the critical factors has seriously impeded the wide range of applications of 3D technology. Electrocardiograph (ECG) and electrooculogram (EOG) have been widely used for monitoring visual fatigue. In this paper, one more objective and effective visual fatigue evaluation model is proposed. Subjective scores (SS), heart rates (HR), blink frequency (BF), Sympathetic activity, Vagal activity, Sympathetic-vagal ratio (a measure of autonomic balance), Average of NN intervals (NN-MEAN) and Standard deviation of NN intervals (SDNN) of all subjects were collected to analyze the change of visual fatigue during the continuous viewing 3D/2D movie. The results showed that SS, HR, BF, Sympathetic activity and SDNN all increase with visual fatigue while NN-MEAN and Vagal activity decrease. As shown by the result of subjective scoring, the visual fatigue has an overall trend of increasing when viewing both 2D and 3D videos and it is higher in 3D condition than in 2D. Based on the results above, two models were built to predict visual fatigue from above indicates during continuous viewing 2D and 3D video processes respectively. The performance of the models makes a good prediction of visual fatigue.

关键词： Visual fatigue Modeling ECG EOG Sympathetic activity Vagal activity

An Adaptive Bionic Sensor: Enhancing Ankle Joint Tracking with High Sensitivity and Superior Cushioning Performance

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Jin, Jianqiao Zhang, Chen Zhao, Jianyuan Yu, Minghan Lei, Ming Jin, Chun Yin, Rui Zhao, Weiwei Sauvage Laboratory for Smart Materials The School of Integrated Circuit Harbin Institute of Technology Shenzhen518055 China Shenzhen Key Laboratory of Flexible Printed Electronics Technology Harbin Institute of Technology Shenzhen518055 China State Key Laboratory of Advanced Welding & Joining Harbin Institute of Technology Harbin150001 China Human-Computer Interaction Design Lab School of System Design and Intelligent Manufacturing Southern University of Science and Technology Shenzhen518055 China

Flexible sensors are renowned for their rapid responsiveness, high flexibility, and outstanding mechanical properties, making them ideal for applications in wearable devices, sports and health monitoring, and human-machine interfaces. To enhance the sensing performance of these flexible sensors, researchers have drawn inspiration from nature, creating various bionic microstructures. However, these structures often manifest in the macroscopic form of thin films, which do little to improve impact resistance and joint protection. In response, this work achieves a highly sensitive and impact-resistant pressure sensor by integrating bionic principles at both micro and macro levels. Specifically, at the micro level, a muscle-like hydrogel system consisting of MXene nanosheets incorporated into the double network of polyvinyl alcohol and sodium alginate matrix is selected. At the macro level, the bionic prototypes of mimosa leaves and durian spikes with good energy absorption structure, together with octopus’ sucker with good adsorption capacity are selected, and the synergistic construction of the three structures is realized through 3D printing methods. The obtained sensor has an appropriate response range, sensitivity and stability. More importantly, the displacement deformation of the hydrogel can be reduced by 91.22%, while its adhesion strength can be increased by 57.5% compared to the unstructured hydrogel. As a proof-of-concept, the proposed bionic flexible sensor can be used to monitor the motion status of the human ankle joint in real-time, thereby assisting users in making real-time judgments on gait health. © 2024, The Authors. All rights reserved.

关键词： Sodium alginate