检索结果-内蒙古大学图书馆

Textessence: A tool for interactive analysis of semantic shifts between corpora

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Newman-Griffis, Denis Sivaraman, Venkatesh Perer, Adam Fosler-Lussier, Eric Hochheiser, Harry Department of Biomedical Informatics University of Pittsburgh United States Human-Computer Interaction Institute Carnegie Mellon University United States Department of Computer Science and Engineering Ohio State University United States

Embeddings of words and concepts capture syntactic and semantic regularities of language;however, they have seen limited use as tools to study characteristics of different corpora and how they relate to one another. We introduce TextEssence, an interactive system designed to enable comparative analysis of corpora using embeddings. TextEssence includes visual, neighbor-based, and similarity-based modes of embedding analysis in a lightweight, web-based interface. We further propose a new measure of embedding confidence based on nearest neighborhood overlap, to assist in identifying high-quality embeddings for corpus analysis. A case study on COVID-19 scientific literature illustrates the utility of the system. TextEssence is available from https://***/drgriffis/text-essence. Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

Working Paper: NLP in human Rights Research - Extracting Knowledge Graphs About Police and Army Units and Their Commanders

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Bauer, Daniel Longley, Tom Ma, Yueen Tony, Wilson Department of Computer Science Columbia University Security Force Monitor Human Rights Institute Columbia Law School

In this working paper we explore the use of an NLP system to assist the work of Security Force Monitor (SFM). SFM creates data about the organizational structure, command personnel and operations of police, army and other security forces, which assists human rights researchers, journalists and litigators in their work to help identify and bring to account specific units and personnel alleged to have committed abuses of human rights and international criminal law. This working paper presents an NLP system that extracts from English language news reports the names of security force units and the biographical details of their personnel, and infers the formal relationship between them. Published alongside this working paper are the system's code and training dataset. We find that the experimental NLP system performs the task at a fair to good level. Its performance is sufficient to justify further development into a live workflow that will give insight into whether its performance translates into savings in time and resource that would make it an effective technical intervention. © 2022, CC BY.

关键词： Natural language processing systems

Head-wise Shareable Attention for Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cao, Zouying Yang, Yifei Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University China Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University China Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3 China

Large Language Models (LLMs) suffer from huge number of parameters, which restricts their deployment on edge devices. Weight sharing is one promising solution that encourages weight reuse, effectively reducing memory usage with less performance drop. However, current weight sharing techniques primarily focus on small-scale models like BERT and employ coarse-grained sharing rules, e.g., layer-wise. This becomes limiting given the prevalence of LLMs and sharing an entire layer or block obviously diminishes the flexibility of weight sharing. In this paper, we present a perspective on head-wise shareable attention for large language models. We further propose two memory-efficient methods that share parameters across attention heads, with a specific focus on LLMs. Both of them use the same dynamic strategy to select the shared weight matrices. The first method directly reuses the pre-trained weights without retraining, denoted as DirectShare. The second method first post-trains with constraint on weight matrix similarity and then shares, denoted as PostShare. Experimental results reveal our head-wise shared models still maintain satisfactory capabilities, demonstrating the feasibility of fine-grained weight sharing applied to LLMs1 Copyright © 2024, The Authors. All rights reserved.

关键词： Matrix algebra

On Improving Boundary Quality of Instance Segmentation in Cluttered and Chaotic Scenarios

学校读者我要写书评

暂无评论

On Improving Boundary Quality of Instance Segmentation in Cl...

IEEE International Conference on Robotics and Automation (ICRA)

作者： Biqi Yang Xiaojie Gao Xianzhi Li Yun-Hui Liu Chi-Wing Fu Pheng-Ann Heng Department of Computer Science and Engineering The Chinese University of Hong Kong Department of Mechanical and Automation Engineering The Chinese University of Hong Kong School of Computer Science and Technology Huazhong University of Science and Technology Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems Chinese Academy of Sciences Shenzhen Institutes of Advanced Technology

Instance segmentation is a long-standing task for supporting robotic bin picking. However, objects of diverse classes can be closely packed with occlusions in cluttered and chaotic scenes, hence, even recent methods could have difficulty in locating clear and precise boundaries to distinguish nearby objects. In this work, we aim to improve the boundary quality of the instance masks for robust and precise instance segmentation in these challenging scenarios. Technical-wise, we first formulate an IoU-based Boundary-aware Mask head (IBM head) for predicting the instance-level mask, boundary, and their corresponding IoU scores. With this core module, we then follow the coarse-to-fine strategy and design our pipeline with two stages: an 1IoUNet to learn localization-based objectness cue and a hierarchical mask refiner to produce sharper and cleaner boundaries. We deploy the IBM head throughout the framework. Extensive experimental results on three grasping benchmarks manifest that our method attains the best instance segmentation performance, compared with the state-of-the-art approaches. Practically, we conduct real-world picking tests to show that with the objectness and boundary IoU scores as guidance, we are able to filter invalid (occluded) instances and select high-fidelity (exposed) instances for grasping.

关键词：

On the Perception of Small Sub-graphs

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Miller, Jacob Ghoniem, Mohammad Wu, Hsiang-Yun Purchase, Helen C. Department of Computer Science University of Arizona United States Luxembourg Institute of Science and Technology Luxembourg St. Pölten University of Applied Sciences Austria Department of Human-Centred Computing University of Monash Australia

Interpreting a node-link graph is enhanced if similar sub-graphs (or ‘motifs’) are depicted in a similar manner – that is, they have the same visual form. Small motifs within graphs may be perceived to be identical when they are structurally dissimilar, or may be perceived to be dissimilar when they are identical. This issue primarily relates to the Gestalt principle of similarity, but may also include an element of quick, low-level pattern-matching. We believe that if motifs are identical, they should be depicted identically;if they are nearly-identical, they should be depicted nearly-identically. This principle is particularly important in domains where motifs hold meaning and where their identification is important. We identified five small motifs: bi-cliques, cliques, cycles, double-cycles, and stars. For each, we defined visual variations on two dimensions – same or different structure, same or different shape. We conducted a crowd-sourced empirical study to test the perception of similarity of these varied motifs, and found that determining whether motifs are identical or similar is affected by both shape and structure. © 2023, CC BY.

关键词： Pattern matching

Saliency Region Detection in Complex Scenes Based on Multi-scale Cascaded Attention

学校读者我要写书评

暂无评论

Saliency Region Detection in Complex Scenes Based on Multi-s...

IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS)

作者： Yifeng Wang Zhenfeng Huo Aoli Liu Lin Zhao Di Wang The Key Laboratory of Smart Human-Computer Interaction and Wearable Technology of Shanxi Province Xidian University China School of Computer Science and Engineering Nanjing University of Science and Technology China

ISBN: (纸本)9781665441506

This paper proposes a saliency detection method based on multi-scale cascade attention mechanism. It utilizes both channel and spatial weight attention mechanism to effectively learn the salient regions. By generating multi-scale intermediate feature maps, the shallow features are divided into categories of foreground and background. Then, the channel weights are calculated by using the foreground and background feature distribution, and the spatial weights are computed by using the predicted feature map, so that the network is more focused on salient regions and suppresses the interference of background regions. Experimental results show that the model can reliably and accurately detect salient targets and delivers better performance.

关键词： Training Cloud computing Conferences Computational modeling Interference Reliability Convolutional neural networks

Distinguishing Non-natural from Natural Adversarial Samples for More Robust Pre-trained Language Model

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Jiayi Bao, Rongzhou Zhang, Zhuosheng Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai200240 China

Recently, the problem of robustness of pretrained language models (PrLMs) has received increasing research interest. Latest studies on adversarial attacks achieve high attack success rates against PrLMs, claiming that PrLMs are not robust. However, we find that the adversarial samples that PrLMs fail are mostly non-natural and do not appear in reality. We question the validity of current evaluation of robustness of PrLMs based on these non-natural adversarial samples and propose an anomaly detector to evaluate the robustness of PrLMs with more natural adversarial samples. We also investigate two applications of the anomaly detector: (1) In data augmentation, we employ the anomaly detector to force generating augmented data that are distinguished as non-natural, which brings larger gains to the accuracy of PrLMs. (2) We apply the anomaly detector to a defense framework to enhance the robustness of PrLMs. It can be used to defend all types of attacks and achieves higher accuracy on both adversarial samples and compliant samples than other defense frameworks. The code is available at https://***/LilyNLP/Distinguishing-Non-Natural. Copyright © 2022, The Authors. All rights reserved.

关键词： Network security

CapsEnhancer: An Effective Computational Framework for Identifying Enhancers Based on Chaos Game Representation and Capsule Network

学校读者我要写书评

暂无评论

Journal of Chemical Information and Modeling 2024年第14期64卷 5725-5736页

作者： Yao, Lantian Xie, Peilin Guan, Jiahui Chung, Chia-Ru Huang, Yixian Pang, Yuxuan Wu, Huacong Chiang, Ying-Chih Lee, Tzong-Yi Kobilka Institute of Innovative Drug Discovery School of Medicine The Chinese University of HongKong Shenzhen518172 China School of Science and Engineering The Chinese University of HongKong Shenzhen518172 China School of Medicine The Chinese University of HongKong Shenzhen518172 China Department of Computer Science and Information Engineering National Central University Taoyuan320317 Taiwan Division of Health Medical Intelligence Human Genome Center The University of Tokyo Tokyo108-8639 Japan Institute of Bioinformatics and Systems Biology National Yang Ming Chiao Tung University Hsinchu300093 Taiwan National Yang Ming Chiao Tung University Hsinchu300093 Taiwan

Enhancers are a class of noncoding DNA, serving as crucial regulatory elements in governing gene expression by binding to transcription factors. The identification of enhancers holds paramount importance in the field of biology. However, traditional experimental methods for enhancer identification demand substantial human and material resources. Consequently, there is a growing interest in employing computational methods for enhancer prediction. In this study, we propose a two-stage framework based on deep learning, termed CapsEnhancer, for the identification of enhancers and their strengths. CapsEnhancer utilizes chaos game representation to encode DNA sequences into unique images and employs a capsule network to extract local and global features from sequence "images". Experimental results demonstrate that CapsEnhancer achieves state-of-the-art performance in both stages. In the first and second stages, the accuracy surpasses the previous best methods by 8 and 3.5%, reaching accuracies of 94.5 and 95%, respectively. Notably, this study represents the pioneering application of computer vision methods to enhancer identification tasks. Our work not only contributes novel insights to enhancer identification but also provides a fresh perspective for other biological sequence analysis tasks. © 2024 The Authors. Published by American Chemical Society.

关键词： DNA sequences

Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewriting

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Mouhammad, Nina Daxenberger, Johannes Schiller, Benjamin Habernal, Ivan DIPF Leibniz Institute for Research and Information in Education Germany Summetix GmbH Germany Trustworthy Human Language Technologies Department of Computer Science Technical University of Darmstadt Germany

Most tasks in NLP require labeled data. Data labeling is often done on crowdsourcing platforms due to scalability reasons. However, publishing data on public platforms can only be done if no privacy-relevant information is included. Textual data often contains sensitive information like person names or locations. In this work, we investigate how removing personally identifiable information (PII) as well as applying differential privacy (DP) rewriting can enable text with privacy-relevant information to be used for crowdsourcing. We find that DP-rewriting before crowdsourcing can preserve privacy while still leading to good label quality for certain tasks and data. PII-removal led to good label quality in all examined tasks, however, there are no privacy guarantees given. © 2023, CC BY.

关键词： Crowdsourcing