检索结果-内蒙古大学图书馆

MindLLM: Lightweight Large language Model Pre-Training, Evaluation and Domain Application

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Yang, Yizhe Sun, Huashan Li, Jiawei Liu, Runheng Li, Yinghao Liu, Yuhang Gao, Yang Huang, Heyan School of Computer Science Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China Beijing Institute of Technology Southeast Academy of Information Technology Fujian Putian China

Large language Models (LLMs) have demonstrated remarkable performance across various natural language tasks, marking significant strides towards general artificial intelligence. While general artificial intelligence is leveraged by developing increasingly large-scale models, there could be another branch to develop lightweight custom models that better serve certain domains, taking into account the high cost of training and deploying LLMs and the scarcity of resources. In this paper, we present MindLLM, a novel series of bilingual lightweight large language models, trained from scratch, alleviating such burdens by offering models with 1.3 billion and 3 billion parameters. A thorough account of experiences accrued during large model development is given, covering every step of the process, including data construction, model architecture, evaluation, and applications. Such insights are hopefully valuable for fellow academics and developers. MindLLM consistently matches or surpasses the performance of other open-source larger models on some public benchmarks. We also introduce an innovative instruction tuning framework tailored for smaller models to enhance their capabilities efficiently. Moreover, we explore the application of MindLLM in specific vertical domains such as law and finance, underscoring the agility and adaptability of our lightweight models. © 2024, The Authors. All rights reserved.

关键词： Artificial intelligence

EchoLoc: Accurate Device-Free Hand Localization Using COTS Devices

学校读者我要写书评

暂无评论

EchoLoc: Accurate Device-Free Hand Localization Using COTS D...

International Conference on Parallel processing (ICPP)

作者： Huijie Chen Fan Li Yu Wang School of Computer Science Beijing Institute of Technology Beijing Engineering Research Center for High Volume Language Information Processing and Cloud Computing Applications Beijing China Department of Computer Science College of Computing and Informatics University of North Carolina at Charlotte Charlotte NC USA

ISBN: (纸本)9781509028245

Hand tracking systems are becoming increasingly popular as a fundamental HCI approach. The trajectory of moving hand can be estimated through smoothing the position coordinates collected from continuous localization. Therefore, hand localization is a key component of any hand tracking systems. This paper presents EchoLoc, which locates the human hand by leveraging the speaker array in Commercial Off-The-Shelf (COTS) devices (i.e., a smart phone plugged with a stereo speaker). EchoLoc measures the distance from the hand to the speaker array via the Time Of Flight (TOF) of the chirp. The speaker array and hand yield a unique triangle, therefore, the hand can be localized with triangular geometry. We prototype EchoLoc on iOS as an application, and find it is capable of localization with the average resolution within five centimeters of 73% and three centimeters of 48%.

关键词： Parallel processing

Gleffn: A Global-Local Event Feature Fusion Network for Micro-Expression Recognition

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Guo, Cunhan Huang, Heyan School of Emergency Management Science and Engineering University of Chinese Academy of Science Beijing China Southeast Academy of information Technology Beijing Institute of Technology Putian China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China

Micro-expressions are brief and subtle facial movements that reveal a person’s true emotions. However, the low frame rate of conventional cameras limits the performance of micro-expression recognition (MER) systems. Event-based cameras are a novel technology that offer high frame rate and low latency, making them suitable for MER tasks, but they are hard to obtain. In this article, we propose a local event feature called local count image, which is derived from up-sampled data using the SloMo method. We also design a global-local event feature fusion network that combines local count image and global features extracted by dense optical flow to learn more discriminative features and solve MER tasks. Experiments showed that our method surpasses state-ofthe-art methods on several datasets. The ablation studies confirm the effectiveness of the local count image feature and the random data augmentation. To the best of our knowledge, our work is the first to tackle MER tasks from an event perspective, and it can facilitate the adoption of event cameras and provide some inspiration for future related works. © 2023, The Authors. All rights reserved.

关键词： Cameras

DESIGN AND DEVELOPMENT OF MORPHOLOGICAL ANALYZER FOR TIGRIGNA VERBS USING HYBRID APPROACH

学校读者我要写书评

暂无评论

Research Square

Research Square 2023年

作者： Gebremeskel, Hagos Gebremedhin Chong, Feng Heyan, Huang Beijing Engineering Research Center of High Volume Language Information Processing Cloud Computing Applications Laboratory School of Computer Science and Technology Beijing Institute of Technology China Department of Computer Science Mekelle University Tigray Mekelle231 Ethiopia

Morphological analyzer is the base for various high-level NLP applications such as information retrieval, spell checking, grammar checking, machine translation, speech recognition, POS tagging and automatic sentence construction. This paper is carefully designed for design and analysis of morphological analyzer Tigrigna verbs using hybrid of memory learning and rules based approaches. The experiment have conducted using Python 3 where TiMBL algorithms IB2 and TRIBL2, and Finite State Transducer rules are used. The performance of the system has been evaluated using 10 fold cross validation technique. Testing was conducted using optimized parameter settings for regular verbs and linguistic rules of the Tigrigna language allomorph and phonology for the irregular verbs. The accuracy of the memory based approach with optimized parameters of TiMBL algorithm IB2 and TRIBL2 was 93.24% and 92.31%, respectively. Finally, the hybrid approach had an actual performance of 95.6% using linguistic rules for handling irregular and copula verbs. © 2023, CC BY.

关键词： Machine learning

Capturing Global Structural information in Long Document Question Answering with Compressive Graph Selector Network

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Nie, Yuxiang Huang, Heyan Wei, Wei Mao, Xian-Ling School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China Beijing Institute of Technology Southeast Academy of Information Technology China Huazhong University of Science and Technology China

Long document question answering is a challenging task due to its demands for complex reasoning over long text. Previous works usually take long documents as non-structured flat texts or only consider the local structure in long documents. However, these methods usually ignore the global structure of the long document, which is essential for long-range understanding. To tackle this problem, we propose Compressive Graph Selector Network (CGSN) to capture the global structure in a compressive and iterative manner. Specifically, the proposed model consists of three modules: local graph network, global graph network and evidence memory network. Firstly, the local graph network builds the graph structure of the chunked segment in token, sentence, paragraph and segment levels to capture the short-term dependency of the text. Secondly, the global graph network selectively receives the information of each level from the local graph, compresses them into the global graph nodes and applies graph attention into the global graph nodes to build the long-range reasoning over the entire text in an iterative way. Thirdly, the evidence memory network is designed to alleviate the redundancy problem in the evidence selection via saving the selected result in the previous steps. Extensive experiments show that the proposed model outperforms previous methods on two datasets.1 Copyright © 2022, The Authors. All rights reserved.

关键词： Iterative methods

AttenWalker: Unsupervised Long-Document Question Answering via Attention-based Graph Walking

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Nie, Yuxiang Huang, Heyan Wei, Wei Mao, Xian-Ling School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center High Volume Language Information Processing and Cloud Computing Applications China Beijing Institute of Technology Southeast Academy of Information Technology China Huazhong University of Science and Technology China

Annotating long-document question answering (long-document QA) pairs is time-consuming and expensive. To alleviate the problem, it might be possible to generate long-document QA pairs via unsupervised question answering (UQA) methods. However, existing UQA tasks are based on short documents, and can hardly incorporate long-range information. To tackle the problem, we propose a new task, named unsupervised long-document question answering (ULQA), aiming to generate high-quality long-document QA instances in an unsupervised manner. Besides, we propose AttenWalker, a novel unsupervised method to aggregate and generate answers with long-range dependency so as to construct long-document QA pairs. Specifically, AttenWalker is composed of three modules, i.e., span collector, span linker and answer aggregator. Firstly, the span collector takes advantage of constituent parsing and reconstruction loss to select informative candidate spans for constructing answers. Secondly, by going through the attention graph of a pre-trained long-document model, potentially interrelated text spans (that might be far apart) could be linked together via an attention-walking algorithm. Thirdly, in the answer aggregator, linked spans are aggregated into the final answer via the mask-filling ability of a pre-trained model. Extensive experiments show that AttenWalker outperforms previous methods on Qasper and NarrativeQA. In addition, AttenWalker also shows strong performance in the few-shot learning setting. Copyright © 2023, The Authors. All rights reserved.

关键词：

Enhancing participant selection through caching in mobile crowd sensing

学校读者我要写书评

暂无评论

Enhancing participant selection through caching in mobile cr...

International Workshop on Quality of Service

作者： Hanshang Li Ting Li Fan Li Weichao Wang Yu Wang Department of Computer Science University of North Carolina at Charlotte Charlotte NC USA Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China School of Computer Science Beijing Institute of Technology Beijing China

With the rapid increasing of smart phones and their embedded sensing technologies, mobile crowd sensing (MCS) becomes an emerging sensing paradigm for performing large-scale sensing tasks. One of the key challenges of large-scale mobile crowd sensing systems is how to effectively select the minimum set of participants from the huge user pool to perform the tasks and achieve certain level of coverage. In this paper, we introduce a new MCS architecture which leverages the cached sensing data to fulfill partial sensing tasks in order to reduce the size of selected participant set. We present a newly designed participant selection algorithm with caching and evaluate it via extensive simulations with a real-world mobile dataset.

关键词： Sensors Mobile communication Memory Algorithm design and analysis Smart phones Probabilistic logic Data models

Building Knowledge-Grounded Dialogue Systems with Graph-Based Semantic Modeling

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Yizhe Huang, Heyan Gao, Yang Li, Jiawei School of Computer Science and Technology Beijing Institute of Technology Beijing100081 China Southeast Academy of Information Technology Beijing Institute of Technology Fujian Putian351100 China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China

The knowledge-grounded dialogue task aims to generate responses that convey information from given knowledge documents. However, it is a challenge for the current sequence-based model to acquire knowledge from complex documents and integrate it to perform correct responses without the aid of an explicit semantic structure. To address these issues, we propose a novel graph structure, Grounded Graph (G2), that models the semantic structure of both dialogue and knowledge to facilitate knowledge selection and integration for knowledge-grounded dialogue generation. We also propose a Grounded Graph Aware Transformer (G2AT) model that fuses multi-forms knowledge (both sequential and graphic) to enhance knowledge-grounded response generation. Our experiments results show that our proposed model outperforms the previous state-of-the-art methods with more than 10% gains in response generation and nearly 20% improvement in factual consistency. Further, our model reveals good generalization ability and robustness. By incorporating semantic structures as prior knowledge in deep neural networks, our model provides an effective way to aid language generation. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

Dynamic Prefix-Tuning for Generative Template-based Event Extraction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Xiao Huang, Heyan Shi, Ge Wang, Bo School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China Key Laboratory of Intelligent Information Processing and Information Security Ministry of Industry and Information Technology China Southeast Academy of Information Technology Beijing Institute of Technology China Faculty of Information Technology Beijing University of Technology China

We consider event extraction in a generative manner with template-based conditional generation. Although there is a rising trend of casting the task of event extraction as a sequence generation problem with prompts, these generation-based methods have two significant challenges, including using suboptimal prompts and static event type information. In this paper, we propose a generative template-based event extraction method with dynamic prefix (GTEE-DYNPREF) by integrating context information with type-specific prefixes to learn a context-specific prefix for each context. Experimental results show that our model achieves competitive results with the state-of-the-art classification-based model ONEIE on ACE 2005 and achieves the best performances on ERE. Additionally, our model is proven to be portable to new types of events effectively. Copyright © 2022, The Authors. All rights reserved.

关键词： Extraction