检索结果-内蒙古大学图书馆

Word-level BERT-CNN-RNN Model for Chinese Punctuation Restoration

学校读者我要写书评

暂无评论

Word-level BERT-CNN-RNN Model for Chinese Punctuation Restor...

International Conference on computer and Communications (ICCC)

作者： Zhe Zhang Jie Liu Lihua Chi Xinhai Chen Science and Technology on Parallel and Distributed Processing Laboratory & Laboratory of Software Engineering for Complex Systems National University of Defense Technology Changsha China College of Computer Science and Electronic Engineering Hunan University Changsha China

ISBN: (数字)9781728186351

ISBN: (纸本)9781728186368

Punctuation restoration in speech recognition has a wide range of application scenarios. Despite the widespread success of neural networks methods at performing punctuation restoration for English, there have been only limited attempts for Chinese punctuation restoration. Due to the differences between Chinese and English in terms of grammar and basic semantic units, existing methods for English is not suitable for Chinese punctuation restoration. To tackle this problem, we propose a hybrid model combining the kernel of Bidirectional Encoder Representations from Transformers (BERT), Convolution Neural Network (CNN) and Recurrent Neural Network (RNN). This model employs a flexible structure and special CNN design which can extract word-level features for Chinese language. We compared the performance of the hybrid model with five widely-used punctuation restoration models on the public dataset. Experimental results demonstrate that our hybrid model is simple and efficient. It outperforms other models and achieves an accuracy of 69.1%.

关键词： Recurrent neural networks Computational modeling Bit error rate Semantics Speech recognition Feature extraction Task analysis

Variational Distillation for Multi-View Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Tian, Xudong Zhang, Zhizhong Wang, Cong Zhang, Wensheng Qu, Yanyun Ma, Lizhuang Wu, Zongze Xie, Yuan Tao, Dacheng School of Computer Science and Technology East China Normal University Shanghai200062 China The Distributed and Parallel Software Laboratory 2012 Labs Huawei Technologies Hangzhou China Institute of Automation Chinese Academy of Sciences Beijing100190 China School of Information Science and Technology Xiamen University Fujian361005 China School of Computer Science and Techology East China Normal University Shanghai China The School of Electronic Information and Electrical Engineering Shanghai Jiao Tong University China College of Mechatronics and Control Engineering Shenzhen University Shenzhen China JD Exploer Academy China The University of Sydney Australia

Information Bottleneck (IB) based multi-view learning provides an information theoretic principle for seeking shared information contained in heterogeneous data descriptions. However, its great success is generally attributed to estimate the multivariate mutual information which is intractable when the network becomes complicated. Moreover, the representation learning tradeoff, i.e., prediction-compression and sufficiency-consistency tradeoff, makes the IB hard to satisfy both requirements simultaneously. In this paper, we design several variational information bottlenecks to exploit two key characteristics (i.e., sufficiency and consistency) for multi-view representation learning. Specifically, we propose a Multi-View Variational Distillation (MV2D) strategy to provide a scalable, flexible and analytical solution to fitting MI by giving arbitrary input of viewpoints but without explicitly estimating it. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels, producing predictive and compact representations naturally. Also, our information-theoretic constraint can effectively neutralize the sensitivity to heterogeneous data by eliminating both task-irrelevant and view-specific information, preventing both tradeoffs in multiple view cases. To verify our theoretically grounded strategies, we apply our approaches to various benchmarks under three different applications. Extensive experiments to quantitatively and qualitatively demonstrate the effectiveness of our approach against state-of-the-art methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

CRAWL: A Trace Routing Algorithm Based on Hybrid Two-Layer Topology 33rd

学校读者我要写书评

暂无评论

CRAWL: A Trace Routing Algorithm Based on Hybrid Two-Layer T...

33rd International Conference on Advanced Information Networking and Applications, AINA 2019

作者： Zheng, Li-ming Long, Wen-feng Liu, Yu-Jia Sun, Wei-dong Department of Message Communication Armed Police Officer Academy Chengdu China National Key Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology Changsha China

ISBN: (纸本)9783030150341

Data distribution is a key technology for resources convergence and sharing in distributed environment. To better meet the requirement for real time data distribution in the dynamic network, a trace routing algorithm named CRAWL based on the hybrid two-layered topology is put forward. The algorithm contains an overlay topology named CBDLO, upper of which consists of multiple distributed balanced binary trees corresponding to different properties and the lower of which is an unstructured topology. CRAWL forwards the data on the lower unstructured topology in the form of random walk, so that the data can be sent to the corresponding upper topology entry. It also includes a matching algorithm named CDM for the parallel matching data properties on the upper distributed and balanced binary tree and transmitting the matched data to the nodes that are interested in the data. The experimental results show that the algorithm can effectively support large-scale data distribution in a dynamical network, reduce distribution overhead and matching delays. © 2019, Springer Nature Switzerland AG.

关键词： Routing algorithms

Multi-agent Fault-tolerant Reinforcement Learning with Noisy Environments

学校读者我要写书评

暂无评论

Multi-agent Fault-tolerant Reinforcement Learning with Noisy...

International Conference on parallel and distributed Systems (ICPADS)

作者： Canhui Luo Xuan Liu Xinning Chen Juan Luo College of Computer Science and Electronic Engineering Hunan University P. R. China The Science and Technology on Parallel and Distributed Processing Laboratory (PDL) P. R. China

ISBN: (数字)9781728190747

ISBN: (纸本)9781728183824

Multi-agent reinforcement learning system is used to solve the problem that agents achieve specific goals in the interaction with the environment through learning policies. Almost all existing multi-agent reinforcement learning methods assume that the observation of the agents is accurate during the training process. It does not take into account that the observation may be wrong due to the complexity of the actual environment or the existence of dishonest agents, which will make the agent training difficult to succeed. In this paper, considering the limitations of the traditional multi-agent algorithm framework in noisy environments, we propose a multi-agent fault-tolerant reinforcement learning (MAFTRL) algorithm. Our main idea is to establish the agent's own error detection mechanism and design the information communication medium between agents. The error detection mechanism is based on the autoencoder, which calculates the credibility of each agent's observation and effectively reduces the environmental noise. The communication medium based on the attention mechanism can significantly improve the ability of agents to extract effective information. Experimental results show that our approach accurately detects the error observation of the agent, which has good performance and strong robustness in both the traditional reliable environment and the noisy environment. Moreover, MAFTRL significantly outperforms the traditional methods in the noisy environment.

关键词： Training Fault tolerance Working environment noise Fault tolerant systems Reinforcement learning Robustness Noise measurement

Profiling HPC Applications with Low Overhead and High Accuracy

学校读者我要写书评

暂无评论

Profiling HPC Applications with Low Overhead and High Accura...

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Jingyuan Zhao Xin Liu Yao Liu Penglong Jiao Jinshuo Liu Wei Xue School of Data Science & Engineering East China Normal University Shanghai China School of Cyber Science and Engineering Wuhan University Wuhan China National Research Centre of Parallel Computer Engineering and Technology Beijing China Shanghai Key Laboratory of Multidimensional Information Processing East China Normal University Shanghai China Tsinghua University Beijing China

As the parallel scale of HPC applications represented by earth system models becomes larger and the computing cost becomes higher, the performance of HPC applications is increasingly critical. Profiling HPC applications accurately helps to model the applications and find the performance bottlenecks. However, due to the complexity of HPC applications, the diversity of programming languages, the differences of individual programming habits, and multiple architectures, accurate profiling becomes very tough. In this paper, we propose LPerf: a low-overhead and high-accuracy profiler for HPC applications. To reduce the profiling overhead and improve the profiling accuracy, we propose a preprocessing method which can automatically instrument with tunable granularity thus significantly reducing the run-time overhead of profiling, an aggregated caller-callee relationship which is used to locate relationship of functions efficiently, and a profiling-aware method which can precisely calculate running time of functions. The experimental results show that the error rate of profiling reaches 0.02%, and the overhead reaches 1.6%, in the earth system model named CAS-ESM. Compared with the baselines, the precision, accuracy, and overhead of LPerf have reached the state of the art.

关键词： Earth Analytical models Schedules computer languages Costs Error analysis Computational modeling

BCS-Net: Boundary, Context and Semantic for Automatic COVID-19 Lung Infection Segmentation from CT Images

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cong, Runmin Yang, Haowei Jiang, Qiuping Gao, Wei Li, Haisheng Wang, Cong Zhao, Yao Kwong, Sam The Institute of Information Science Beijing Jiaotong University Beijing100044 China Beijing Key Laboratory of Big Data Technology for Food Safety Beijing Technology and Business University Beijing100048 China The Department of Computer Science City University of Hong Kong Hong Kong The Beijing Key Laboratory of Advanced Information Science and Network Technology Beijing100044 China The School of Information Science and Engineering Ningbo University Ningbo315211 China The School of Electronic and Computer Engineering Peking University Shenzhen Graduate School Shenzhen518055 China Peng Cheng Laboratory Shenzhen518055 China The Distributed and Parallel Software Lab Huawei Technologies Shenzhen518129 China The City University of Hong Kong Shenzhen Research Institute Shenzhen51800 China

The spread of COVID-19 has brought a huge disaster to the world, and the automatic segmentation of infection regions can help doctors to make diagnosis quickly and reduce workload. However, there are several challenges for the accurate and complete segmentation, such as the scattered infection area distribution, complex background noises, and blurred segmentation boundaries. To this end, in this paper, we propose a novel network for automatic COVID-19 lung infection segmentation from CT images, named BCS-Net, which considers the boundary, context, and semantic attributes. The BCS-Net follows an encoder-decoder architecture, and more designs focus on the decoder stage that includes three progressively Boundary-Context-Semantic Reconstruction (BCSR) blocks. In each BCSR block, the attention-guided global context (AGGC) module is designed to learn the most valuable encoder features for decoder by highlighting the important spatial and boundary locations and modeling the global context dependence. Besides, a semantic guidance (SG) unit generates the semantic guidance map to refine the decoder features by aggregating multi-scale high-level features at the intermediate resolution. Extensive experiments demonstrate that our proposed framework outperforms the existing competitors both qualitatively and quantitatively. © 2022, CC BY-NC-SA.

关键词： COVID-19

Secure and Fast Decision Tree Evaluation on Outsourced Cloud Data 2nd

学校读者我要写书评

暂无评论

Secure and Fast Decision Tree Evaluation on Outsourced Cloud...

2nd International Conference on Machine Learning for Cyber Security, ML4CS 2019

作者： Liu, Lin Su, Jinshu Chen, Rongmao Chen, Jinrong Sun, Guangliang Li, Jie School of Computer National University of Defense Technology Changsha China National Key Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha China National University of Defense Technology Changsha China

ISBN: (纸本)9783030306182

Decision trees are famous machine learning classifiers which have been widely used in many areas, such as healthcare, text classification and remote diagnostics, etc. The service providers usually host a decision tree model on the cloud server and provide some classification service for clients to use such a model remotely. In such a scenario, the model is a valuable asset to the cloud which should not be disclosed to the clients, while the query data and classification results are private to the client. To solve such a problem, we propose several building blocks, i.e., secure comparison and secure polynomial calculation, in a two-cloud model. Based on these building blocks, we design a privacy-preserving decision tree evaluation scheme. Compared with the most recent works, our scheme can fully protect the tree model and clients’ data privacy simultaneously. Besides, our scheme also supports offline service users which is essential to the system’s scalability. Moreover, through theoretical analysis and real-world experimental test, it is oblivious that our scheme is quite efficient. © 2019, Springer Nature Switzerland AG.

关键词： Decision trees

Deep reinforcement learning for combinatorial optimization: Covering salesman problems

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Kaiwen Zhang, Tao Wang, Rui Wang, Yuheng Han, Yi College of Systems Engineering National University of Defense Technology Changsha410073 China Hunan Key Laboratory of Multi-Energy System Intelligent Interconnection Technology HKL-MSI2T Changsha410073 China Graduate College National University of Defense Technology Changsha410073 China Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha410073 China

This paper introduces a new deep learning approach to approximately solve the Covering Salesman Problem (CSP). In this approach, given the city locations of a CSP as input, a deep neural network model is designed to directly output the solution. It is trained using the deep reinforcement learning without supervision. Specifically, in the model, we apply the Multi-head Attention to capture the structural patterns, and design a dynamic embedding to handle the dynamic patterns of the problem. Once the model is trained, it can generalize to various types of CSP tasks (different sizes and topologies) with no need of re-training. Through controlled experiments, the proposed approach shows desirable time complexity: it runs more than 20 times faster than the traditional heuristic solvers with a tiny gap of optimality. Moreover, it significantly outperforms the current state-of-the-art deep learning approaches for combinatorial optimization in the aspect of both training and inference. In comparison with traditional solvers, this approach is highly desirable for most of the challenging tasks in practice that are usually large-scale and require quick decisions. Copyright © 2021, The Authors. All rights reserved.

关键词： Combinatorial optimization

Improving unsupervised domain adaptation by reducing bi-level feature redundancy

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Mengzhu Zhang, Xiang Lan, Long Wang, Wei Tan, Huibin Luo, Zhigang Science and Technology on Parallel and Distributed Laboratory College of Computer National University of Defense Technology Changsha China Institute for Quantum State Key Laboratory of High Performance Computing National University of Defense Technology Changsha China DUT-RU International School of Information Science & Engineering Dalian University of Technology DalianLiaoning116000 China Department of Science and Technology on Parallel and Distributed Processing National University of Defense Technology Changsha China

Reducing feature redundancy has shown beneficial effects for improving the accuracy of deep learning models, thus it is also indispensable for the models of unsupervised domain adaptation (UDA). Nevertheless, most recent efforts in the field of UDA ignore this point. Moreover, main schemes realizing this in general independent of UDA purely involve a single domain, thus might not be effective for cross-domain tasks. In this paper, we emphasize the significance of reducing feature redundancy for improving UDA in a bi-level way. For the first level, we try to ensure compact domain-specific features with a transferable decorrelated normalization module, which preserves specific domain information whilst easing the side effect of feature redundancy on the sequel domain-invariance. In the second level, domain-invariant feature redundancy caused by domain-shared representation is further mitigated via an alternative brand orthogonality for better generalization. These two novel aspects can be easily plugged into any BN-based backbone neural networks. Specifically, simply applying them to ResNet50 has achieved competitive performance to the state-of-the-arts on five popular benchmarks. Our code will be available at https://***/dreamkily/gUDA. © 2020, CC BY.

关键词： Deep learning