检索结果-内蒙古大学图书馆

ER-NeRF++: Efficient region-aware Neural Radiance Fields for high-fidelity talking portrait synthesis

Information Fusion 2024年 110卷

作者： Li, Jiahe Zhang, Jiawei Bai, Xiao Zheng, Jin Zhou, Jun Gu, Lin School of Computer Science and Engineering State Key Laboratory of Complex & Critical Software Environment Jiangxi Research Institute Beihang University Beijing China School of Information and Communication Technology Griffith University Brisbane Australia RIKEN AIP Tokyo Japan The University of Tokyo Tokyo Japan

Despite conditional Neural Radiance Fields (NeRF) achieving great success in modeling audio-driven talking portraits, the generation quality is increasingly hampered by the lack of efficient use of space information. This paper presents ER-NeRF, a novel conditional NeRF-based architecture for talking portrait synthesis, and its variant version ER-NeRF++ to concurrently achieve fast convergence, real-time rendering, and state-of-the-art performance with small model size. Inspired by the unequal contribution of spatial regions, we propose two modules in ER-NeRF to guide the talking portrait modeling: (1) A compact and expressive Tri-Plane Hash Representation to improve the accuracy of dynamic head reconstruction by pruning empty spatial regions with three planar hash encoders. (2) A Region Attention Module for the audio–visual feature fusion, including a novel cross-modal attention mechanism to connect audio features with different spatial regions explicitly for local motion priors. Additionally, to tackle the difficulty in learning large facial motions, we propose a deformable variant ER-NeRF++ by including a Deformation Grid Transformer to enable the reuse of cross-regional spatial features for large motion representation. Compared to ER-NeRF, our ER-NeRF++ framework achieves a significant improvement in facial motion quality while maintaining the ability of fast training and real-time rendering. For the torso part, a directAdaptive Pose Encoding is introduced to simplify the pose information for a better head-torso connection. Extensive experiments demonstrate that both of our proposed frameworks can efficiently render lifelike talking portrait videos with rich realistic details, performing better in image quality and audio-lip synchronization compared to previous methods. © 2024 Elsevier B.V.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

Principled representation learning for entity alignment

arXiv

引用

arXiv 2021年

作者： Guo, Lingbing Sun, Zequn Chen, Mingyang Hu, Wei Zhang, Qiang Chen, Huajun College of Computer Science and Technology Zhejiang University Hangzhou China State Key Laboratory for Novel Software Technology Nanjing University Nanjing China

Embedding-based entity alignment (EEA) has recently received great attention. Despite significant performance improvement, few efforts have been paid to facilitate understanding of EEA methods. Most existing studies rest on the assumption that a small number of pre-aligned entities can serve as anchors connecting the embedding spaces of two KGs. Nevertheless, no one investigates the rationality of such an assumption. To fill the research gap, we define a typical paradigm abstracted from existing EEA methods and analyze how the embedding discrepancy between two potentially aligned entities is implicitly bounded by a predefined margin in the scoring function. Further, we find that such a bound cannot guarantee to be tight enough for alignment learning. We mitigate this problem by proposing a new approach, named NeoEA, to explicitly learn KG-invariant and principled entity embeddings. In this sense, an EEA model not only pursues the closeness of aligned entities based on geometric distance, but also aligns the neural ontologies of two KGs by eliminating the discrepancy in embedding distribution and underlying ontology knowledge. Our experiments demonstrate consistent and significant improvement in performance against the best-performing EEA methods. Copyright © 2021, The Authors. All rights reserved.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

Centralized Feature Pyramid for Object Detection

arXiv

引用

arXiv 2022年

作者： Quan, Yu Zhang, Dong Zhang, Liyan Tang, Jinhui The School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing210094 China The College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing211106 China

Visual feature pyramid has shown its superiority in both effectiveness and efficiency in a wide range of applications. However, the existing methods exorbitantly concentrate on the inter-layer feature interactions but ignore the intra-layer feature regulations, which are empirically proved beneficial. Although some methods try to learn a compact intra-layer feature representation with the help of the attention mechanism or the vision transformer, they ignore the neglected corner regions that are important for dense prediction tasks. To address this problem, in this paper, we propose a Centralized Feature Pyramid (CFP) for object detection, which is based on a globally explicit centralized feature regulation. Specifically, we first propose a spatial explicit visual center scheme, where a lightweight MLP is used to capture the globally long-range dependencies and a parallel learnable visual center mechanism is used to capture the local corner regions of the input images. Based on this, we then propose a globally centralized regulation for the commonly-used feature pyramid in a top-down fashion, where the explicit visual center information obtained from the deepest intra-layer feature is used to regulate frontal shallow features. Compared to the existing feature pyramids, CFP not only has the ability to capture the global long-range dependencies, but also efficiently obtain an all-round yet discriminative feature representation. Experimental results on the challenging MS-COCO validate that our proposed CFP can achieve the consistent performance gains on the state-of-the-art YOLOv5 and YOLOX object detection baselines. The code has been released at: CFPNet. Copyright © 2022, The Authors. All rights reserved.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

SEMICON: A Learning-to-hash Solution for Large-scale Fine-grained Image Retrieval

arXiv

引用

arXiv 2022年

作者： Shen, Yang Sun, Xuhao Wei, Xiu-Shen Jiang, Qing-Yuan Yang, Jian School of Computer Science and Engineering Nanjing University of Science and Technology China State Key Laboratory of Integrated Services Networks Xidian University China State Key Laboratory for Novel Software Technology Nanjing University China Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security Nanjing University of Science and Technology China

In this paper, we propose Suppression-Enhancing Mask based attention and Interactive Channel transformatiON (SEMICON) to learn binary hash codes for dealing with large-scale fine-grained image retrieval tasks. In SEMICON, we first develop a suppression-enhancing mask (SEM) based attention to dynamically localize discriminative image regions. More importantly, different from existing attention mechanism simply erasing previous discriminative regions, our SEM is developed to restrain such regions and then discover other complementary regions by considering the relation between activated regions in a stage-by-stage fashion. In each stage, the interactive channel transformation (ICON) module is afterwards designed to exploit correlations across channels of attended activation tensors. Since channels could generally correspond to the parts of fine-grained objects, the part correlation can be also modeled accordingly, which further improves fine-grained retrieval accuracy. Moreover, to be computational economy, ICON is realized by an efficient two-step process. Finally, the hash learning of our SEMICON consists of both global- and local-level branches for better representing fine-grained objects and then generating binary hash codes explicitly corresponding to multiple levels. Experiments on five benchmark fine-grained datasets show our superiority over competing methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Image retrieval

来源：评论

学校读者我要写书评

暂无评论

Realizing Emotional Interactions to Learn User Experience and Guide Energy Optimization for Mobile Architectures 22

Realizing Emotional Interactions to Learn User Experience an...

引用

Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture

作者： Xueliang Li Zhuobin Shi Junyang Chen Yepang Liu National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China College of Computer Science and Software Engineering Shenzhen University China Research Institute of Trustworthy Autonomous Systems Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation and Department of Computer Science and Engineering Southern University of Science and Technology China

ISBN: (纸本)9781665462723

In the age of AI, mobile architectures such as smartphones are still "cold machines"; machines do not feel. If the architecture is able to feel users' feelings and runtime user experience (UX), it will accordingly adapt performance/energy to find the optimal system-operating state that consumes the least energy to satisfy users. In this paper, we will utilize users' facial expressions (FEs) to learn their runtime UX. We know that FEs are the natural and direct way for humans to convey their emotions and feelings. Our study reveals that FEs also reflect UX. Our research for the first time quantifies the link between FEs and UX. Leveraging this link, the architecture will be able to use the front camera to see FEs and feel users' UX. Based on UX, the architecture can appropriately provision computing resources. We propose Vi-energy system to realize the above idea. Our evaluation shows that Vi-energy reduces energy consumption by 52.9% at maximum and secures UX.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Streaming Hierarchical Clustering Based on Point-Set Kernel

Research Square

引用

Research Square 2022年

作者： Han, Xin Zhu, Ye Ting, Kai Ming Zhan, De-Chuan Li, Gang School of Computer Science Xi’an Shiyou University China School of Information Technology Deakin University Australia State Key Laboratory of Novel Software Technology Nanjing University China Centre for Cyber Security Research and Innovation Deakin University Australia

Hierarchical clustering produces a cluster tree with different granularities. As a result, hierarchical clustering provides richer information and insight into a dataset than partitioning clustering. However, hierarchical clustering algorithms often have two weaknesses: scalability and the capacity to handle clusters of varying densities. This is because they rely on pairwise point-based similarity calculations and the similarity measure is independent of data distribution. In this paper, we aim to overcome these weaknesses and propose a novel efficient hierarchical clustering called StreaKHC that enables massive streaming data to be mined. The enabling factor is the use of a scalable point-set kernel to measure the similarity between an existing cluster in the cluster tree and a new point in the data stream. It also has an efficient mechanism to update the hierarchical structure so that a high-quality cluster tree can be maintained in real-time. Our extensive empirical evaluation shows that StreaKHC is more accurate and more efficient than existing hierarchical clustering algorithms. © 2022, CC BY.

关键词： Trees (mathematics)

来源：评论

学校读者我要写书评

暂无评论

A Reference Architecture for Blockchain-based Traceability Systems Using Domain-Driven Design and Microservices

A Reference Architecture for Blockchain-based Traceability S...

引用

Asia-Pacific Conference on software Engineering

作者： Yanze Wang Shanshan Li Huikun Liu He Zhang Bo Pan State Key Laboratory of Novel Software Technology Software Institute Nanjing University Nanjing China Terminal Cloud Service Department Huawei Technologies Co.Ltd Nanjing China

ISBN: (纸本)9781665455381

Traceability systems are important for solving problems due to the increasing scale of the global supply chain, such as food safety crises and market disorder. Blockchain, as an immutable and decentralized ledger, is able to optimize the traditional traceability system by ensuring the transparency and reliability of the system data. However, the use of blockchain technology may lead to a rapid increase in the complexity of system design and development. It is challenging to address widespread and complicated business, changeable processes, and massive data in practice, which are the main factors restricting the wide application of a blockchain-based traceability system (BTS). Therefore, in this paper, we reviewed relevant studies and proposed a reference architecture for BTSs. The proposed reference architecture can improve the cohesiveness, maintainability, and extensibility of BTSs through domain-driven design (DDD) and microservices. Considering the efficiency reduction caused by massive data and complicated data structure, we further changed the traditional single blockchain framework into multiple subchain networks, which could improve development efficiency and system performance. With the guidance of the architecture trade-off analysis method (ATAM), we evaluated our reference architecture and implemented a prototype in the salmon supply chain scenario. The results show that our solution is effective and adaptable to meet the requirements of BTSs.

关键词： System performance Supply chains Microservice architectures Prototypes Data structures Extensibility Blockchains

来源：评论

学校读者我要写书评

暂无评论

Financial Trading Decisions based on Deep Fuzzy Self-Organizing Map

arXiv

引用

arXiv 2022年

作者： Dehao, Pei Chao, Luo School of Information Science and Engineering Shandong Normal University Jinan250014 China Shandong Provincial Key Laboratory for Novel Distributed Computer Software Technology Jinan250014 China

The volatility features of financial data would considerably change in different periods, that is one of the main factors affecting the applications of machine learning in quantitative trading. Therefore, to effectively distinguish fluctuation patterns of financial markets can provide meaningful information for the trading decision. In this article, a novel intelligent trading system based on deep fuzzy self-organizing map (DFSOM) companied with GRU networks is proposed, where DFSOM is utilized for the clustering of financial data to acquire multiple fluctuation patterns in an unsupervised way. Firstly, in order to capture the trend features and evade the effect of high noises in financial data, the images of extended candlestick charts instead of raw data are processed and the obtained features are applied for the following unsupervised learning, where candlestick charts are produced with both price and volume information. Secondly, by using the candlestick features, a two-layer deep fuzzy self-organizing map is constructed to carry out the clustering, where two-layer models carry out the clustering in multiple time scales to improve the processing of time-dependent information. Thirdly, GRU networks are used to implement the prediction task, based on which an intelligent trading model is constructed. The feasibility and effectiveness of the proposed method are verified by using various real financial data sets. Copyright © 2022, The Authors. All rights reserved.

关键词： Self organizing maps

来源：评论

学校读者我要写书评

暂无评论

Patch Synthesis for Property Repair of Deep Neural Networks

arXiv

引用

arXiv 2024年

作者： Chi, Zhiming Ma, Jianan Yang, Pengfei Huang, Cheng-Chao Li, Renjue Wang, Jingyi Huang, Xiaowei Zhang, Lijun Key Laboratory of System Software Chinese Academy of Sciences State Key Laboratory of Computer Science Institute of Software Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China School of cyberspace Hangzhou Dianzi University Hangzhou China Zhejiang University Hangzhou China College of Computer and Information Science Software College Southwest University Chongqing China Nanjing Institute of Software Technology Chinese Academy of Sciences Nanjing China University of Liverpool Liverpool United Kingdom

Deep neural networks (DNNs) are prone to various dependability issues, such as adversarial attacks, which hinder their adoption in safety-critical domains. Recently, NN repair techniques have been proposed to address these issues while preserving original performance by locating and modifying guilty neurons and their parameters. However, existing repair approaches are often limited to specific data sets and do not provide theoretical guarantees for the effectiveness of the repairs. To address these limitations, we introduce PATCHPRO, a novel patch-based approach for property-level repair of DNNs, focusing on local robustness. The key idea behind PATCHPRO is to construct patch modules that, when integrated with the original network, provide specialized repairs for all samples within the robustness neighborhood while maintaining the network’s original performance. Our method incorporates formal verification and a heuristic mechanism for allocating patch modules, enabling it to defend against adversarial attacks and generalize to other inputs. PATCHPRO demonstrates superior efficiency, scalability, and repair success rates compared to existing DNN repair methods, i.e., realizing provable property-level repair for 100% cases across multiple high-dimensional datasets. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Sed-Dedup: An efficient secure deduplication system with data modifications

Sed-Dedup: An efficient secure deduplication system with dat...

引用

作者： Tian, Wenlong Li, Ruixuan Xu, Cheng-Zhong Xu, Zhiyong School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China Electrical and Computer Engineering Wayne State University DetroitMI United States State Key Laboratory of IoTSC and Department of Computer Science University of Macau China Math and Computer Science Department Suffolk University BostonMA United States Shenzhen Institute of Advanced Technology Chinese Academy of Science Shenzhen China

The amount of outsourced data grows rapidly. In recent years, cloud service providers integrate data deduplication systems with convergent encryption (CE) methods, in which a file encryption key is determined by its own content instead of the secret of a specific user, to save the storage cost and ensure the security of outsourced data. However, present secure deduplication systems failed to deal with data modifications efficiently. We observe that when a client makes small changes on an existing file, the current chunking algorithms cannot effectively detect the similarities and always create chunks with largely overlapped contents. It reduces data deduplication ratios and results in unnecessary overhead. In this paper, we propose Sed-Dedup, an efficient secure delta encoding deduplication system to address this problem. In Sed-Dedup, we introduce a novel delta encoding approach to store modified contents in delta files and leave the original files intact. Two schemes with different encoding policies are designed. Both of them can solve the issue and improve the secure deduplication performance. To evaluate the performance, we implement a prototype and conduct extensive experiments based on synthetic and real-world datasets. Our experimental results show that Sed-Dedup is superior to the state-of-the-art secure deduplication systems. © 2019 John Wiley & Sons, Ltd.

关键词： Cryptography

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：