检索结果-内蒙古大学图书馆

ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling 41

学校读者我要写书评

暂无评论

ESM All-Atom: Multi-scale Protein Language Model for Unified...

41st International Conference on Machine Learning, ICML 2024

作者： Zheng, Kangjie Long, Siyu Lu, Tianyu Yang, Junwei Dai, Xinyu Zhang, Ming Nie, Zaiqing Ma, Wei-Ying Zhou, Hao School of Computer Science National Key Laboratory for Multimedia Information Processing Peking University Anker Embodied AI Lab Peking University Beijing100871 China School of Artificial Intelligence National Key Laboratory for Novel Software Technology Nanjing University China Department of Computer Science Tsinghua University China Tsinghua University China PharMolix Inc. China

Protein language models have demonstrated significant potential in the field of protein ***, current protein language models primarily operate at the residue scale, which limits their ability to provide information at the atom *** limitation prevents us from fully exploiting the capabilities of protein language models for applications involving both proteins and small *** this paper, we propose ESM-AA (ESM All-Atom), a novel approach that enables atom-scale and residue-scale unified molecular ***-AA achieves this by pretraining on multi-scale code-switch protein sequences and utilizing a multi-scale position encoding to capture relationships among residues and *** results indicate that ESM-AA surpasses previous methods in protein-molecule tasks, demonstrating the full utilization of protein language *** investigations reveal that through unified molecular modeling, ESM-AA not only gains molecular knowledge but also retains its understanding of proteins. Copyright 2024 by the author(s)

关键词： Molecular docking

Many-to-Few Decomposition: Linking R2-Based and Decomposition-Based Multiobjective Efficient Global Optimization Algorithms

学校读者我要写书评

暂无评论

IEEE Transactions on Evolutionary Computation 2024年 1-1页

作者： Zhao, Liang Huang, Xiaobin Qian, Chao Zhang, Qingfu Department of Computer Science City University of Hong Kong Hong Kong National Key Laboratory for Novel Software Technology Nanjing University China

In multiobjective optimization, the R2 indicator is widely used for designing indicator-based algorithms, and the Tchebycheff approach is commonly employed in decomposition-based algorithms. Despite their wide use, the connection between these two different paradigms is still not well understood, particularly in the field of multiobjective efficient global optimization (MOEGO). Considering that expected improvement (EI) is a cornerstone in efficient global optimization, this paper first studies the relationship between R2-based EI and Tchebycheff-based EI. Then, we introduce a Many-to-Few (M2F) decomposition framework, offering a new perspective for linking the R2-based method and the Tchebycheff decomposition approach. By incorporating M2F decomposition into MOEGO, a new algorithm called R2/D-EGO is proposed. At each iteration, R2/D-EGO utilizes the Tchebycheff decomposition paradigm to generate a set of candidate solutions, each one corresponding to a different weight vector. Subsequently, a subset of query points is selected from the candidates based on the lower bound of R2-based EI. Empirical results indicate that the proposed R2/D-EGO is highly competitive in comparison with both R2-based and decomposition-based MOEGO algorithms in the parallel (or batch) setting. IEEE

关键词： Monte Carlo methods

Context Sketching for Memory-efficient Graph Representation Learning

学校读者我要写书评

暂无评论

Context Sketching for Memory-efficient Graph Representation ...

IEEE International Conference on Data Mining (ICDM)

作者： Kai-Lang Yao Wu-Jun Li Department of Computer Science and Technology National Key Laboratory for Novel Software Technology Nanjing University China

Graph representation learning (GRL) is fundamental in multi-graph applications like molecular property prediction. Graph neural networks (GNNs) have emerged as a popular method for GRL. However, existing GRL methods primarily focus on designing GNN models with enhanced expressiveness while overlooking the memory efficiency of algorithms during training. The memory inefficiency problem is caused by a contextual constraint imposed on node representations, which requires each node to be context-dependent on its input graph. In this paper, we propose a novel method, called context sketching (COS), for memory-efficient graph representation learning in multi-graph scenarios. We first formally define the contextual constraint based on the enclosed $\infty-$hop subgraphs of nodes. Subsequently, we propose to relax the original contextual constraint by requiring each node to be context-dependent on its enclosed $k-$hop subgraph $(k \ll \infty)$ which is a contextual sketch of the enclosed $\infty$-hop subgraph. Lastly, we prove that COS constructs an optimal solution to a memory-related objective associated with graph coarsening. Experiments on four widely used benchmark datasets demonstrate that COS can reduce the memory footprint of baselines by a large margin with almost no accuracy loss.

关键词：

On the Optimal Batch Size for Byzantine-Robust Distributed Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yang, Yi-Rui Shi, Chang-Wei Li, Wu-Jun National Key Laboratory for Novel Software Technology Department of Computer Science and Technology Nanjing University China

Byzantine-robust distributed learning (BRDL), in which computing devices are likely to behave abnormally due to accidental failures or malicious attacks, has recently become a hot research topic. However, even in the independent and identically distributed (i.i.d.) case, existing BRDL methods will suffer from a significant drop on model accuracy due to the large variance of stochastic gradients. Increasing batch sizes is a simple yet effective way to reduce the variance. However, when the total number of gradient computation is fixed, a too-large batch size will lead to a too-small iteration number (update number), which may also degrade the model accuracy. In view of this challenge, we mainly study the optimal batch size when the total number of gradient computation is fixed in this work. In particular, we theoretically and empirically show that when the total number of gradient computation is fixed, the optimal batch size in BRDL increases with the fraction of Byzantine workers. Therefore, compared to the case without attacks, the batch size should be set larger when under Byzantine attacks. However, for existing BRDL methods, large batch sizes will lead to a drop on model accuracy, even if there is no Byzantine attack. To deal with this problem, we propose a novel BRDL method, called Byzantine-robust stochastic gradient descent with normalized momentum (ByzSGDnm), which can alleviate the drop on model accuracy in large-batch cases. Moreover, we theoretically prove the convergence of ByzSGDnm for general non-convex cases under Byzantine attacks. Empirical results show that ByzSGDnm has a comparable performance to existing BRDL methods under bit-flipping failure, but can outperform existing BRDL methods under deliberately crafted attacks. Copyright © 2023, The Authors. All rights reserved.

关键词： Drops

GNNFORMER: A GRAPH-BASED FRAMEWORK FOR CYTOPATHOLOGY REPORT GENERATION

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Yang-Fan Yao, Kai-Lang Li, Wu-Jun National Key Laboratory for Novel Software Technology Department of Computer Science and Technology Nanjing University China

Cytopathology report generation is a necessary step for the standardized examination of pathology images. However, manually writing detailed reports brings heavy workloads for pathologists. To improve efficiency, some existing works have studied automatic generation of cytopathology reports, mainly by applying image caption generation frameworks with visual encoders originally proposed for natural images. A common weakness of these works is that they do not explicitly model the structural information among cells, which is a key feature of pathology images and provides significant information for making diagnoses. In this paper, we propose a novel graph-based framework called GNNFormer, which seamlessly integrates graph neural network (GNN) and Transformer into the same framework, for cytopathology report generation. To the best of our knowledge, GNNFormer is the first report generation method that explicitly models the structural information among cells in pathology images. It also effectively fuses structural information among cells, fine-grained morphology features of cells and background features to generate high-quality reports. Experimental results on the NMI-WSI dataset show that GNNFormer can outperform other state-of-the-art baselines. Copyright © 2023, The Authors. All rights reserved.

关键词： Cells

Controllable Financial Market Generation with Diffusion Guided Meta Agent

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Yu-Hao Xu, Chang Liu, Yang Liu, Weiqing Li, Wu-Jun Bian, Jiang National Key Laboratory for Novel Software Technology Department of Computer Science and Technology Nanjing University China Microsoft Research Asia China

Order flow modeling stands as the most fundamental and essential financial task, as orders embody the minimal unit within a financial market. However, current approaches often result in unsatisfactory fidelity in generating order flow, and their generation lacks controllability, thereby limiting their application scenario. In this paper, we advocate incorporating controllability into the market generation process, and propose a Diffusion Guided meta Agent (DiGA) model to address the problem. Specifically, we utilize a diffusion model to capture dynamics of market state represented by time-evolving distribution parameters about mid-price return rate and order arrival rate, and define a meta agent with financial economic priors to generate orders from the corresponding distributions. Extensive experimental results demonstrate that our method exhibits outstanding controllability and fidelity in generation. Furthermore, we validate DiGA’s effectiveness as generative environment for downstream financial applications. © 2024, CC BY.

关键词： Commerce

Validity-Preserving Delta Debugging via Generator Trace Reduction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ren, Luyao Zhang, Xing Hua, Ziyue Jiang, Yanyan He, Xiao Xiong, Yingfei Xie, Tao Key Laboratory of High Confidence Software Technologies Ministry of Education School of Computer Science Peking University Beijing China State Key Laboratory for Novel Software Technology Department of Computer Science and Technology Nanjing University China School of Computer and Communication Engineering University of Science and Technology Beijing China

Reducing test inputs that trigger bugs is crucial for efficient debugging. Delta debugging is the most popular approach for this purpose. When test inputs need to conform to certain specifications, existing delta debugging practice encounters a validity problem: it blindly applies reduction rules, producing a large number of invalid test inputs that do not satisfy the required specifications. This overall diminishing effectiveness and efficiency becomes even more pronounced when the specifications extend beyond syntactical structures. Our key insight is that we should leverage input generators, which are aware of these specifications, to generate valid reduced inputs, rather than straightforwardly performing reduction on test inputs. In this paper, we propose a generator-based delta debugging method, namely GReduce, which derives validity-preserving reducers. Specifically, given a generator and its execution, demonstrating how the bug-inducing test input is generated, GReduce searches for other executions on the generator that yield reduced, valid test inputs. The evaluation results on five benchmarks (i.e., graphs, DL models, JavaScript programs, SymPy, and algebraic data types) show that GReduce substantially outperforms state-of-the-art syntax-based reducers including Perses and T-PDD, and also outperforms QuickCheck, SmartCheck, as well as the state-of-the-art choice-sequence-based reducer Hypothesis, demonstrating the effectiveness, efficiency, and versatility of GReduce. Copyright © 2024, The Authors. All rights reserved.

关键词： software testing

FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yang, Yi-Rui Wang, Kun Li, Wu-Jun National Key Laboratory for Novel Software Technology Department of Computer Science and Technology Nanjing University China

Federated learning (FL) has recently become a hot research topic, in which Byzantine robustness, communication efficiency and privacy preservation are three important aspects. However, the tension among these three aspects makes it hard to simultaneously take all of them into account. In view of this challenge, we theoretically analyze the conditions that a communication compression method should satisfy to be compatible with existing Byzantine-robust methods and privacy-preserving methods. Motivated by the analysis results, we propose a novel communication compression method called consensus sparsification (ConSpar). To the best of our knowledge, ConSpar is the first communication compression method that is designed to be compatible with both Byzantine-robust methods and privacy-preserving methods. Based on ConSpar, we further propose a novel FL framework called FedREP, which is Byzantine-robust, communication-efficient and privacy-preserving. We theoretically prove the Byzantine robustness and the convergence of FedREP. Empirical results show that FedREP can significantly outperform communication-efficient privacy-preserving baselines. Furthermore, compared with Byzantine-robust communication-efficient baselines, FedREP can achieve comparable accuracy with the extra advantage of privacy preservation. Copyright © 2023, The Authors. All rights reserved.

关键词： Privacy-preserving techniques

LW2G: Learning Whether to Grow for Prompt-based Continual Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Feng, Qian Zhou, Da-Wei Zhao, Hanbin Zhang, Chao Qian, Hui College of Computer Science and Technology Zhejiang University Hangzhou China State Key Laboratory for Novel Software Technology Nanjing University Nanjing China

Continual Learning (CL) aims to learn in non-stationary scenarios, progressively acquiring and maintaining knowledge from sequential tasks. Recent Prompt-based Continual Learning (PCL) has achieved remarkable performance with Pre-Trained Models (PTMs). These approaches grow a prompt sets pool by adding a new set of prompts when learning each new task (prompt learning) and adopt a matching mechanism to select the correct set for each testing sample (prompt retrieval). Previous studies focus on the latter stage by improving the matching mechanism to enhance Prompt Retrieval Accuracy (PRA). To promote cross-task knowledge facilitation and form an effective and efficient prompt sets pool, we propose a plug-in module in the former stage to Learn Whether to Grow (LW2G) based on the disparities between tasks. Specifically, a shared set of prompts is utilized when several tasks share certain commonalities, and a new set is added when there are significant differences between the new task and previous tasks. Inspired by Gradient Projection Continual Learning, our LW2G develops a metric called Hinder Forward Capability (HFC) to measure the hindrance imposed on learning new tasks by surgically modifying the original gradient onto the orthogonal complement of the old feature space. With HFC, an automated scheme Dynamic Growing Approach adaptively learns whether to grow with a dynamic threshold. Furthermore, we design a gradient-based constraint to ensure the consistency between the updating prompts and pre-trained knowledge, and a prompts weights reusing strategy to enhance forward transfer. Extensive experiments show the effectiveness of our method. The source codes are available at https://***/RAIAN08/LW2G. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning