检索结果-内蒙古大学图书馆

SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hu, Pengbo Li, Xingyu Zhou, Yi School of Mathematical Sciences University of Science and Technology of China Hefei China Shanghai Center for Brain Science and Brain-Inspired Technology Shanghai China National Engineering Laboratory for Brain-inspired Intelligence Technology and Application School of Information Science and Technology University of Science and Technology of China Hefei China

As deep learning advances, there is an ever-growing demand for models capable of synthesizing information from multi-modal resources to address the complex tasks raised from real-life applications. Recently, many large multi-modal datasets have been collected, on which researchers actively explore different methods of fusing multimodal information. However, little attention has been paid to quantifying the contribution of different modalities within the proposed models. In this paper, we propose the SHapley vAlue-based PErceptual (SHAPE) scores that measure the marginal contribution of individual modalities and the degree of cooperation across modalities. Using these scores, we systematically evaluate different fusion methods on different multi-modal datasets for different tasks. Our experiments suggest that for some tasks where different modalities are complementary, the multi-modal models still tend to use the dominant modality alone and ignore the cooperation across modalities. On the other hand, models learn to exploit cross-modal cooperation when different modalities are indispensable for the task. In this case, the scores indicate it is better to fuse different modalities at relatively early stages. We hope our scores can help improve the understanding of how the present multi-modal models operate on different modalities and encourage more sophisticated methods of integrating multiple modalities. © 2022, CC BY.

关键词： Large dataset

Amyloidβneurotoxicity is IDO1–Kyn–AhR dependent and blocked by IDO1 inhibitor

学校读者我要写书评

暂无评论

Signal Transduction and Targeted Therapy 2020年第1期5卷 1633-1635页

作者： Zhenzhen Duan Shengnan Zhang Heng Liang Zikang Xing Leilei Guo Lei Shi Lisha Du Chunxiang Kuang Osamu Takikawa Qing Yang State Key Laboratory of Genetic Engineering School of Life SciencesInstitute of Science and Technology for Brain-Inspired IntelligenceMOE Engineering Research Center of Gene TechnologyShanghai Engineering Research Center of Industrial MicroorganismsFudan UniversityHandan Road 220Shanghai 200433China School of Chemical Science and Engineering Tongji UniversitySiping Road 1239Shanghai 200092China National Institute for Longevity Sciences National Center for Geriatrics and Gerontology35 GengoMoriokaObuAichi 474-8511Japan

Dear Editor,Alzheimer’s disease(AD),the most common neurodegenerative disorder and a leading cause of dementia is characterized by progressive memory deficits,cognitive impairment and personality *** accumulation of extracellular amyloid beta(Aβ)plaques consisted by Aβpeptide and intracellular neurofibrillary tangles(NFTs)composed by misfolded hyperphosphorylated tau are main pathological hallmarks of AD,which ultimately lead to the dysfunction and loss of synapses and the eventual death of *** plays a causal role in AD *** the mechanism underlying neuroinflammationinduced AD pathology has not been fully elucidated,increasing evidence proposed that the kynurenine pathway(KP),a major route for L-tryptophan(L-Trp)catabolism,plays an important role.

关键词： Aβ tau Alzheimer

Task-independent knowledge makes for transferable representations for generalized zero-shot learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Chaoqun Chen, Xuejin Min, Shaobo Sun, Xiaoyan Li, Houqiang School of Data Science University of Science and Technology of China HefeiAnhui China National Engineering Laboratory for Brain-inspired Intelligence Technology and Application University of Science and Technology of China HefeiAnhui China

Generalized Zero-Shot Learning (GZSL) targets recognizing new categories by learning transferable image representations. Existing methods find that, by aligning image representations with corresponding semantic labels, the semantic-aligned representations can be transferred to unseen categories. However, supervised by only seen category labels, the learned semantic knowledge is highly task-specific, which makes image representations biased towards seen categories. In this paper, we propose a novel Dual-Contrastive Embedding Network (DCEN) that simultaneously learns task-specific and task-independent knowledge via semantic alignment and instance discrimination. First, DCEN leverages task labels to cluster representations of the same semantic category by cross-modal contrastive learning and exploring semantic-visual complementarity. Besides task-specific knowledge, DCEN then introduces task-independent knowledge by attracting representations of different views of the same image and repelling representations of different images. Compared to high-level seen category supervision, this instance discrimination supervision encourages DCEN to capture low-level visual knowledge, which is less biased toward seen categories and alleviates the representation bias. Consequently, the task-specific and task-independent knowledge jointly make for transferable representations of DCEN, which obtains averaged 4.1% improvement on four public benchmarks. © 2021, CC BY.

关键词： Semantics

Dual progressive prototype network for generalized zero-shot learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Chaoqun Min, Shaobo Chen, Xuejin Sun, Xiaoyan Li, Houqiang School of Data Science The National Engineering Laboratory for Brain-inspired Intelligence Technology and Application University of Science and Technology of China Anhui Hefei China Tencent Data Platform Guangdong Shenzhen China

Generalized Zero-Shot Learning (GZSL) aims to recognize new categories with auxiliary semantic information, e.g., category attributes. In this paper, we handle the critical issue of domain shift problem, i.e., confusion between seen and unseen categories, by progressively improving cross-domain transferability and category discriminability of visual representations. Our approach, named Dual Progressive Prototype Network (DPPN), constructs two types of prototypes that record prototypical visual patterns for attributes and categories, respectively. With attribute prototypes, DPPN alternately searches attribute-related local regions and updates corresponding attribute prototypes to progressively explore accurate attribute-region correspondence. This enables DPPN to produce visual representations with accurate attribute localization ability, which benefits the semantic-visual alignment and representation transferability. Besides, along with progressive attribute localization, DPPN further projects category prototypes into multiple spaces to progressively repel visual representations from different categories, which boosts category discriminability. Both attribute and category prototypes are collaboratively learned in a unified framework, which makes visual representations of DPPN transferable and distinctive. Experiments on four benchmarks prove that DPPN effectively alleviates the domain shift problem in GZSL. Copyright © 2021, The Authors. All rights reserved.

关键词： Semantics

Exploiting Correspondences with All-pairs Correlations for Multi-view Depth Estimation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cheng, Kai Chen, Hao Yin, Wei Xu, Guangkai Chen, Xuejin The National Engineering Laboratory for Brain Inspired Intelligence Technology and Application University of Science and Technology of China Hefei230026 China The Computer Graphics and Computer Assisted Design Laboratory Zhejiang University Hangzhou310058 China DJI Technology

Multi-view depth estimation plays a critical role in reconstructing and understanding the 3D world. Recent learning-based methods have made significant progress in it. However, multi-view depth estimation is fundamentally a correspondence-based optimization problem, but previous learning-based methods mainly rely on predefined depth hypotheses to build correspondence as the cost volume and implicitly regularize it to fit depth prediction, deviating from the essence of iterative optimization based on stereo correspondence. Thus, they suffer unsatisfactory precision and generalization capability. In this paper, we are the first to explore more general image correlations to establish correspondences dynamically for depth estimation. We design a novel iterative multi-view depth estimation framework mimicking the optimization process, which consists of 1) a correlation volume construction module that models the pixel similarity between a reference image and source images as all-to-all correlations;2) a flow-based depth initialization module that estimates the depth from the 2D optical flow;3) a novel correlation-guided depth refinement module that reprojects points in different views to effectively fetch relevant correlations for further fusion and integrate the fused correlation for iterative depth update. Without predefined depth hypotheses, the fused correlations establish multi-view correspondence in an efficient way and guide the depth refinement heuristically. We conduct sufficient experiments on ScanNet, DeMoN, ETH3D, and 7Scenes to demonstrate the superiority of our method on multi-view depth estimation and its best generalization ability. Copyright © 2022, The Authors. All rights reserved.

关键词： Optimization

DDDM: a brain-inspired Framework for Robust Classification

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chen, XiYuan Li, Xingyu Zhou, Yi Yang, Tianming Institute of Neuroscience Center for Excellence in Brain Science and Intelligence Technology Chinese Academy of Sciences Shanghai China University of Chinese Academy of Sciences Beijing China Shanghai Center for Brain Science and Brain-Inspired Technology Shanghai China National Engineering Laboratory for Brain-Inspired Intelligence Technology and Application School of Information Science and Technology University of Science and Technology of China Hefei China

Despite their outstanding performance in a broad spectrum of real-world tasks, deep artificial neural networks are sensitive to input noises, particularly adversarial perturbations. On the contrary, human and animal brains are much less vulnerable. In contrast to the one-shot inference performed by most deep neural networks, the brain often solves decision-making with an evidence accumulation mechanism that may trade time for accuracy when facing noisy inputs. The mechanism is well described by the Drift-Diffusion Model (DDM). In the DDM, decision-making is modeled as a process in which noisy evidence is accumulated toward a threshold. Drawing inspiration from the DDM, we propose the Dropout-based Drift-Diffusion Model (DDDM) that combines test-phase dropout and the DDM for improving the robustness for arbitrary neural networks. The dropouts create temporally uncorrelated noises in the network that counter perturbations, while the evidence accumulation mechanism guarantees a reasonable decision accuracy. Neural networks enhanced with the DDDM tested in image, speech, and text classification tasks all significantly outperform their native counterparts, demonstrating the DDDM as a task-agnostic defense against adversarial attacks. © 2022, CC BY.

关键词： Deep neural networks

Multi-Modal Optimization of Stacked Chip Macro-Module Layout Based on Memristor-inspired Evolutionary Game 4

学校读者我要写书评

暂无评论

Multi-Modal Optimization of Stacked Chip Macro-Module Layout...

4th International Conference on Electronic Information engineering and Computer Communication, EIECC 2024

作者： Qiao, Kaikai Ma, Ben Chen, Mingze Ding, Ke Li, Changqing Duan, Shukai Wang, Lidan Southwest University College of Artificial Intelligence Chongqing China College of Artificial Intelligence Southwest University Chongqing China National Local Joint Engineering Research Center of of Intelligent Transmission and Control Technology Chongqing China Chongqing Key Laboratory of Brain-inspired Computing and Intelligent Chips Chongqing China Ministry of Education Chongqing China

ISBN: (纸本)9798331534622

As the complexity of chip design continues to increase, the stacking of multiple device layers in a three dimension(3D) architecture has emerged as a promising approach, which aims to optimize the placement and interconnections of large function blocks across the vertically-stacked device layers. Due to the complex interactions between layout and design constraints, traditional deterministic optimization often struggle to navigate its multi-modal search landscape effectively. Thus, we creatively propose the memristive evolutionary game: the diverse metal layers engage in a strategic interplay that determines the ownership hierarchy of chip macro modules and the 2D layout of these chip macro modules within each designated metal layer is orchestrated using evolutionary algorithm, which harness the power of game theoretic evolution to seek the optimal solution, balancing the objectives of minimizing layout area and wiring length. Upon comparison with alternative algorithms, the game evolution algorithm introduced in this paper exhibits superior optimization performance. © 2024 IEEE.

关键词： Memristors

3D Colon Reconstruction with Self-Supervised Depth Estimation and Global Registration

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Cheng, Kai Xu, Hanmo Ma, Yiting Sun, Bin Li, Yang Chen, Xuejin The National Engineering Laboratory for Brain-inspired Intelligence Technology and Application University of Science and Technology of China Hefei230026 China Institute of Artificial Intelligence Hefei230026 China The First Affiliated Hospital of Anhui Medical University Hefei230026 China

Background and Objective: Reconstructing 3D colon models can assist in diagnosis and treatment during colonoscopy, such as the analysis of polyp size and morphology, and the visualization of missing regions during a colonoscopy. However, 3D reconstruction algorithm of colonoscopy is a challenging task due to colon’s textureless and non-Lambertian surfaces, and poor conditions resulting from manual operations, such as limited field of vision caused by cameras that move very close along colon surfaces, and motion blur. In this paper, we propose a 3D reconstruction method for colonoscopy with self-supervised depth estimation and global ***: A novel self-supervised depth estimation method is first proposed for colonoscopy images to boost 3D colon reconstruction. We utilize synthetic data to learn representative appearance features for depth estimation and exploit depth consistency among neighboring frames for self-supervision in real videos. Then we design DepthSLAM, a simultaneous localization-and-mapping (SLAM) module that integrates depth priors for accurate and scaleconsistent colon reconstruction. To further improve the stability of reconstructing entire colons from long videos, we propose a segment-wise reconstruction pipeline with an interactive global registration module to compose segments as a complete model with the reference of a common colon *** and Conclusions: Experiments on public datasets UCL colonoscopy and C3VD demonstrate the superiority of our depth estimation method. Extensive experiments on colonoscopy videos from the C3VD dataset, the ColonSimulation dataset, and the LDPolypVideo dataset demonstrate that our method can achieve accurate localization and reconstruction on both colon chunks and complete models. Besides, ablation studies on C3VD and Colon-Simulation show that the proposed depth priors and registration module can improve the accuracy of localization. To the best of our knowledge, we are the first to a

关键词： Textures

LAENet: Light-weight asymmetric encoder-decoder network for semantic segmentation

学校读者我要写书评

暂无评论

LAENet: Light-weight asymmetric encoder-decoder network for ...

2021 International Conference on Artificial intelligence and Industrial technology applications, AIITA 2021

作者： Hong, Liangyi Duan, Shukai Wang, Lidan Pan, Yongbin College of Electronic and Information Engineering Southwest University Chongqing China College of Artificial Intelligence Southwest University Chongqing China Brain-inspired Computing and Intelligent Control Chongqing Key Lab Chongqing China National and Local Joint Engineering Laboratory of Intelligent Transmission and Control Technology Chongqing China Chongqing Brain Science Collaborative Innovation Center Chongqing China

Encode-decoder structure is used in deep learning for real-time dense segmentation task. On account of the limitation of calculation burden on mobile devices, we present a light-weight asymmetric encoder-decoder network in this paper, namely LAENet, which quickly and efficiently accomplish the task of real-time semantic segmentation. We employ an asymmetric convolution and group convolution structure combined with dilated convolution and dense connectivity to reduce computation cost and model size, which can guarantee adequate receptive field and enhance the model learning ability in encoder. On the other hand, feature pyramid networks (FPN) structure combine attention mechanism and ECRE block are utilized in the decoder to strike a balance between the network complexity and segmentation performance. Our approach achieves only have 0.84M parameters, and is able to reach 66 FPS in a single GTX 1080Ti GPU. Experiments on Cityscapes datasets demonstrate that superior performance of LAENet is better than the existing segmentation network, in terms of speed and accuracy trade-off without any post-processing. © Published under licence by IOP Publishing Ltd.

关键词： Convolution