检索结果-内蒙古大学图书馆

Uncovering Neural Scaling Laws in Molecular Representation Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Dingshuo Zhu, Yanqiao Zhang, Jieyu Du, Yuanqi Li, Zhixun Liu, Qiang Wu, Shu Wang, Liang Center for Research on Intelligent Perception and Computing Institute of Automation Chinese Academy of Sciences China Department of Computer Science University of California Los Angeles United States The Paul G. Allen School of Computer Science and Engineering University of Washington United States Department of Computer Science Cornell University United States Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong Hong Kong

Molecular Representation Learning (MRL) has emerged as a powerful tool for drug and materials discovery in a variety of tasks such as virtual screening and inverse design. While there has been a surge of interest in advancing model-centric techniques, the influence of both data quantity and quality on molecular representations is not yet clearly understood within this field. In this paper, we delve into the neural scaling behaviors of MRL from a data-centric viewpoint, examining four key dimensions: (1) data modalities, (2) dataset splitting, (3) the role of pre-training, and (4) model capacity. Our empirical studies confirm a consistent power-law relationship between data volume and MRL performance across these dimensions. Additionally, through detailed analysis, we identify potential avenues for improving learning efficiency. To challenge these scaling laws, we adapt seven popular data pruning strategies to molecular data and benchmark their performance. Our findings underline the importance of data-centric MRL and highlight possible directions for future research. Copyright © 2023, The Authors. All rights reserved.

关键词： Benchmarking

TD-NET: A TRI-DOMAIN NETWORK FOR SPARSE-VIEW CT RECONSTRUCTION

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Xinyuan Su, Changqing Xiong, Bo University College London Department of Computer Science LondonWC1E 6BT United Kingdom Peking University National Engineering Research Center of Visual Technology Beijing100871 China

Sparse-view CT reconstruction, aimed at reducing X-ray radiation risks, frequently suffers from image quality degradation, manifested as noise and artifacts. Existing post-processing and dual-domain techniques, although effective in radiation reduction, often lead to over-smoothed results, compromising diagnostic clarity. Addressing this, we introduce TD-Net, a pioneering tri-domain approach that unifies sinogram, image, and frequency domain optimizations. By incorporating Frequency Supervision Module(FSM), TD-Net adeptly preserves intricate details, overcoming the prevalent over-smoothing issue. Extensive evaluations demonstrate TD-Net’s superior performance in reconstructing high-quality CT images from sparse views, efficiently balancing radiation safety and image fidelity. The enhanced capabilities of TD-Net in varied noise scenarios highlight its potential as a breakthrough in medical imaging. Copyright © 2023, The Authors. All rights reserved.

关键词： Medical imaging

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fang, Luyang Yu, Xiaowei Cai, Jiazhang Chen, Yongkai Wu, Shushan Liu, Zhengliang Yang, Zhenyuan Lu, Haoran Gong, Xilin Liu, Yufang Ma, Terry Ruan, Wei Abbasi, Ali Zhang, Jing Wang, Tao Latif, Ehsan Liu, Wei Zhang, Wei Kolouri, Soheil Zhai, Xiaoming Zhu, Dajiang Zhong, Wenxuan Liu, Tianming Ma, Ping Department of Statistics University of Georgia GA United States Department of Computer Science and Engineering The University of Texas ArlingtonTX United States Department of Statistics Harvard University MA United States School of Computing University of Georgia GA United States School of Computer Science Carnegie Mellon University PA United States Department of Computer Science Vanderbilt University TN United States AI4STEM Education Center University of Georgia GA United States Department of Radiation Oncology Mayo Clinic Arizona AZ United States School of Computer and Cyber Sciences Augusta University GA United States

The exponential growth of Large Language Models (LLMs) continues to highlight the need for efficient strategies to meet ever-expanding computational and data demands. This survey provides a comprehensive analysis of two complementary paradigms: Knowledge Distillation (KD) and Dataset Distillation (DD), both aimed at compressing LLMs while preserving their advanced reasoning capabilities and linguistic diversity. We first examine key methodologies in KD, such as task-specific alignment, rationale-based training, and multi-teacher frameworks, alongside DD techniques that synthesize compact, high-impact datasets through optimization-based gradient matching, latent space regularization, and generative synthesis. Building on these foundations, we explore how integrating KD and DD can produce more effective and scalable compression strategies. Together, these approaches address persistent challenges in model scalability, architectural heterogeneity, and the preservation of emergent LLM abilities. We further highlight applications across domains such as healthcare and education, where distillation enables efficient deployment without sacrificing performance. Despite substantial progress, open challenges remain in preserving emergent reasoning and linguistic diversity, enabling efficient adaptation to continually evolving teacher models and datasets, and establishing comprehensive evaluation protocols. By synthesizing methodological innovations, theoretical foundations, and practical insights, our survey charts a path toward sustainable, resource-efficient LLMs through the tighter integration of KD and DD principles. © 2025, CC BY-NC-ND.

关键词： Teaching

Logical Relation Modeling and Mining in Hyperbolic Space for Recommendation

学校读者我要写书评

暂无评论

Logical Relation Modeling and Mining in Hyperbolic Space for...

International Conference on Data Engineering

作者： Yanchao Tan Hang Lv Zihao Zhou Wenzhong Guo Bo Xiong Weiming Liu Chaochao Chen Shiping Wang Carl Yang College of Computer and Data Science Fuzhou University Fuzhou China Engineering Research Center of Big Data Intelligence Ministry of Education Fuzhou China Fujian Key Laboratory of Network Computing and Intelligent Information Processing Fuzhou University Fuzhou China Institute for Artificial Intelligence University of Stuttgart Stuttgart Germany College of Computer Science Zhejiang University Hangzhou China Department of Computer Science Emory University Atlanta United States

ISBN: (数字)9798350317152

ISBN: (纸本)9798350317169

The sparse interactions between users and items have aggravated the difficulty of their representations in recommender systems. Existing methods leverage tags to alleviate the sparsity problem but ignore prevalent logical relations among items and tags (e.g., membership, hierarchy, and exclusion), which can be leveraged to enhance the accuracy of modeling user preferences and conducting recommendations. To this end, we propose to extract logical relations among item tags from existing tag taxonomies and exploit the individual strengths of the Poincaré and the Lorentz models in hyperbolic space for logical relation modeling towards enhanced recommendations. Moreover, we find that the logical relations directly extracted from existing tag taxonomies can be inaccurate and coarse. Therefore, we further devise innovative consistency-based and granularity- based weighting mechanisms based on user behavior patterns for data-driven logical relation mining that can be jointly optimized along with recommendations in an end-to-end fashion. Extensive experiments on four real-world benchmark datasets show drastic performance gains brought by our proposed framework, which constantly achieves an average of 8.25% improvement over state-of-the-art competitors regarding both Recall and NDCG metrics. Insightful case studies further demonstrate that our automatically refined logical relations are highly accurate and interpretable.

关键词： Measurement Accuracy Taxonomy Semantics Predictive models Performance gain Linguistics

Video Mobile-Former: Video Recognition with Efficient Global Spatial-temporal Modeling

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Rui Wu, Zuxuan Chen, Dongdong Chen, Yinpeng Dai, Xiyang Liu, Mengchen Zhou, Luowei Yuan, Lu Jiang, Yu-Gang Shanghai Key Lab of Intelligent Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Microsoft Cloud&AI China

Transformer-based models have achieved top performance on major video recognition benchmarks. Benefiting from the self-attention mechanism, these models show stronger ability of modeling long-range dependencies compared to CNN-based models. However, significant computation overheads, resulted from the quadratic complexity of self-attention on top of a tremendous number of tokens, limit the use of existing video transformers in applications with limited resources like mobile devices. In this paper, we extend Mobile-Former to Video Mobile-Former, which decouples the video architecture into a lightweight 3D-CNNs for local context modeling and a Transformer modules for global interaction modeling in a parallel fashion. To avoid significant computational cost incurred by computing self-attention between the large number of local patches in videos, we propose to use very few global tokens (e.g., 6) for a whole video in Transformers to exchange information with 3D-CNNs with a cross-attention mechanism. Through efficient global spatial-temporal modeling, Video Mobile-Former significantly improves the video recognition performance of alternative lightweight baselines, and outperforms other efficient CNN-based models at the low FLOP regime from 500M to 6G total FLOPs on various video recognition tasks. It is worth noting that Video Mobile-Former is the first Transformer-based video model which constrains the computational budget within 1G FLOPs. Copyright © 2022, The Authors. All rights reserved.

关键词： Benchmarking

READFake: Reflection and Environment-Aware DeepFake Detection

学校读者我要写书评

暂无评论

READFake: Reflection and Environment-Aware DeepFake Detectio...

IEEE visual Communications and Image Processing (VCIP)

作者： Muhammad Mohzary Elham Basunduwah Sejun Song Baek-Young Choi Department of Computer Science Jazan University Jazan Saudi Arabia Engineering and Technology Research Center Jazan University Jazan Saudi Arabia School of Science and Engineering University of Missouri-Kansas City MO USA College of Engineering and Computing in Al-Qunfudhah Umm Al-Qura University Makkah Saudi Arabia School of Computer and Cyber Science Augusta University Augusta GA USA

ISBN: (数字)9798331529543

ISBN: (纸本)9798331529550

This paper presents a novel Reflection and Environment-Aware DeepFake (READFake) detection technique. Using reflections on various body parts (e.g., eyes, nose, cheeks, etc.) and environmental factors, we validate the hypothesis that the existing DeepFake creation methods, including reenactment, replacement, and synthesis, fail to coordinate their counterfeits with the reflective components along with the given environmental mapping. We detect various features from the specular highlight images, including color components, shapes, and textures, to check the coordination with the surrounding environmental factors, such as indoor/outdoor, bright/dark backgrounds, and light strength. We have conducted extensive experiments to evaluate the performance of READFake using various input parameters and advanced Deep Neural Network (DNN) architectures on multiple public DeepFake datasets. The empirical results show that READFake achieves high accuracy (99.00%) in detecting sophisticated DeepFake images.

关键词： Deepfakes Accuracy Shape Image color analysis visual communication Nose Artificial neural networks Feature extraction Reflection Environmental factors

Metaverse: A Vision, Architectural Elements, and Future Directions for Scalable and Realtime Virtual Worlds

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ismail, Leila Buyya, Rajkumar Lab School of Computing and Information Systems The University of Melbourne Australia Lab Department of Computer Science and Software Engineering National Water and Energy Center United Arab Emirates University United Arab Emirates

With the emergence of Cloud computing, Internet of Things-enabled Human-computer Interfaces, Generative Artificial Intelligence, and high-accurate Machine and Deep-learning recognition and predictive models, along with the Post Covid-19 proliferation of social networking, and remote communications, the Metaverse gained a lot of popularity. Metaverse has the prospective to extend the physical world using virtual and augmented reality so the users can interact seamlessly with the real and virtual worlds using avatars and holograms. It has the potential to impact people in the way they interact on social media, collaborate in their work, perform marketing and business, teach, learn, and even access personalized healthcare. Several works in the literature examine Metaverse in terms of hardware wearable devices, and virtual reality gaming applications. However, the requirements of realizing the Metaverse in realtime and at a large-scale need yet to be examined for the technology to be usable. To address this limitation, this paper presents the temporal evolution of Metaverse definitions and captures its evolving requirements. Consequently, we provide insights into Metaverse requirements. In addition to enabling technologies, we lay out architectural elements for scalable, reliable, and efficient Metaverse systems, and a classification of existing Metaverse applications along with proposing required future research *** Codes ***, *** © 2023, CC BY.

关键词： Energy efficiency

Computational Methods for Analysing Biometric Systems 3

学校读者我要写书评

暂无评论

Computational Methods for Analysing Biometric Systems

3rd International Conference on Technological Advancements in Computational sciences, ICTACS 2023

作者： Multani, Fahim Bijlwan, Sheela Upadhyay, Viyat Varun Goyal, Bhawna Dogra, Ayush Lepcha, Dawa Chyophel Ies College of Technology Bhopal Department of Computer Science Engineering Madhya Pradesh India Uttaranchal School of Computing Sciences Uttaranchal Institute of Technology Uttaranchal University Dehradun248007 India Gla University Department of Mechanical Engineering Mathura Mathura India University Center for Research and Development Chandigarh University Punjab Mohali India Chitkara University Institute of Engineering and Technology Chitkara University Punjab India Chandigarh University Department Electronics and Communication Engineering Punjab Mohali India

ISBN: (纸本)9798350342338

The use of biometric data, such as various anatomical and behavioural attributes in automatic identification and authentication systems for access control, surveillance, computer security, and law enforcement are increasing. Several nations are already or will soon begin to use biometric technology. In this paper, there is a in detail analysis of innovative biometric systems and also the research study of how the computational techniques used intelligent data system in order to analyse biometric system, The efficacy of biometric technologies will depend on how well they work. Biometric system evaluation examines how well biometric systems adapt to particular characteristics. Planning and carrying out biometric tests are frequently handled by an impartial third party. A measure of confidence in the validity of the claim might be one of the options the algorithm offers. Applications that validate a user's address or confirm their identity at the point of sale are examples of verification applications. © 2023 IEEE.

关键词： Attributes Biometric data Computational techniques Innovative Research Technology

SocialEdge: Socialized Learning-Based Request Scheduling for Edge-Cloud Systems

学校读者我要写书评

暂无评论

SocialEdge: Socialized Learning-Based Request Scheduling for...

International Conference on Distributed computing Systems

作者： Ziwei Wang Yunfeng Zhao Chao Qiu Qiang He Xin Wang Xiaofei Wang Qinghua Hu College of Intelligence and Computing Tianjin University Tianjin China Guangdong Laboratory of Artificial Intelligence and DigitalEconomy (SZ) Shenzhen China School of Computer Science and Technology Huazhong University of Science and Technology China Department of Computing Technologies Swinburne University of Technology Australia National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab Huazhong University of Science and Technology Wuhan China

The ability for cloud data centres and edge data centres to collaborate unleashes the potential of the edge-cloud system. However, its sophistication causes unexpected issues in request scheduling, such as Insufficient intelligence, complicated hierarchy and limited cooperation. Traditional resource optimization methods for the edge-cloud system struggle to accommodate such intricate situations. In this paper, inspired by the behaviors and regulations in human society, we propose a socialized learning-based scheduling approach for the edge-cloud system, namely SocialEdge. In order to adapt to time-varying requests and dynamic service prevalence, we propose a learning-based approach, multi-agent advantage actor-critic (MAA2C) and graph convolutional networks-based MAA2C for two phases, respectively, which can improve individual intelligence to achieve optimal dispatch and orchestration decisions. Then, to make use of hierarchy correlation, we develop the socialized refining inter the layers, where inference results from one layer guide the training process of the other layer so that optimization-critical knowledge can be abstracted. Besides, we apply socialized cooperation to each layer, where federated averaging with MAA2C is implemented to ensure cooperation over private data for achieving optimal global solutions. Experimental evaluations on a proof-of-concept testbed along with two real traces demonstrate that baselines have 105% more delay than SocialEdge while being 76.6% less time efficiency and 21.6% less throughput.

关键词：