检索结果-内蒙古大学图书馆

CROSS-ENTROPY IS ALL YOU NEED TO INVERT THE DATA GENERATING PROCESS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Reizinger, Patrik Bizeul, Alice Juhos, Attila Vogt, Julia E. Balestriero, Randall Brendel, Wieland Klindt, David Max Planck Institute for Intelligent Systems Tübingen AI Center ELLIS Institute Tübingen Germany Department of Computer Science ETH Zürich ETH AI Center ETH Zürich Zürich Switzerland Department of Computer Science Brown University RI United States Cold Spring Harbor Laboratory Cold Spring Harbor New York United States

Supervised learning has become a cornerstone of modern machine learning, yet a comprehensive theory explaining its effectiveness remains elusive. Empirical phenomena, such as neural analogy-making and the linear representation hypothesis, suggest that supervised models can learn interpretable factors of variation in a linear fashion. Recent advances in self-supervised learning, particularly nonlinear Independent Component Analysis, have shown that these methods can recover latent structures by inverting the data generating process. We extend these identifiability results to parametric instance discrimination, then show how insights transfer to the ubiquitous setting of supervised learning with cross-entropy minimization. We prove that even in standard classification tasks, models learn representations of ground-truth factors of variation up to a linear transformation under a certain DGP. We corroborate our theoretical contribution with a series of empirical studies. First, using simulated data matching our theoretical assumptions, we demonstrate successful disentanglement of latent factors. Second, we show that on DisLib, a widely-used disentanglement benchmark, simple classification tasks recover latent structures up to linear transformations. Finally, we reveal that models trained on ImageNet encode representations that permit linear decoding of proxy factors of variation. Together, our theoretical findings and experiments offer a compelling explanation for recent observations of linear representations, such as superposition in neural networks. This work takes a significant step toward a cohesive theory that accounts for the unreasonable effectiveness of supervised learning. © 2024, CC BY.

关键词： Self-supervised learning

Decision-Making Framework For AMR Fleet Size In Manufacturing Environments

学校读者我要写书评

暂无评论

Decision-Making Framework For AMR Fleet Size In Manufacturin...

IEEE International Conference on Autonomous Robot systems and Competitions (ICARSC)

作者： Catarina Rema Romão Santos Henrique Piqueiro Diogo Miguel Matos Paulo Moura Oliveirat Pedro Costa Manuel F. Silva Fillipe L. Ribeiro CRIIS - Centre for Robotics in Industry and Intelligent Systems INESC TEC - Institute for Systems and Computer Engineering Technology and Science Porto Portugal Department of Engineering University of Trás-os-Montes and Alto Douro (UTAD) Vila Real Portugal Faculty of Engineering University of Porto (FEUP) Porto Portugal ISEP Polytechnic of Porto rua Dr. António Bernardino de Almeida Porto Portugal JPM Industry Vale de Cambra Portugal

ISBN: (数字)9798331538606

ISBN: (纸本)9798331538613

Industry 4.0 is transforming manufacturing environments, with robotics being a key technology that enhances various capabilities. The flexibility of Autonomous Mobile Robots has led to the rise of multi-robot systems in industrial settings. Considering the high cost of these robots, it is essential to determine the best fit of number and type before making any major investments. Simulation and modeling are valuable decision-support tools, allowing the simulation of different setups to address robot fleet sizing issues. This paper introduces a decision-support framework that combines a fleet manager software stack with the FlexSim simulator, helping decision-makers determine the most suitable mobile robots fleet size tailored to their needs. Unlike previous approaches, the developed solution integrates the same real robot coordination software in both simulation and actual deployment, ensuring that tested scenarios accurately reflect real-world conditions. A case study was conducted to evaluate the framework, involving multiple tasks of loading and unloading materials within a warehouse. Five different scenarios with varying fleet sizes were simulated, and their performances assessed. The analysis concluded that, for the case study under consideration, a fleet of three robots was the most suitable, considering relevant key performance indicators. The results confirmed that the developed solution is an effective alternative for addressing the problem and represents a novel technology with no prior state-of-the-art equivalents.

关键词： Service robots Robot kinematics Decision making Dynamic scheduling Software Batteries Software reliability Resource management Mobile robots Autonomous robots

An Optimized Ensemble Model for Prediction the Bandwidth of Metamaterial Antenna

学校读者我要写书评

暂无评论

computers, Materials & Continua 2022年第4期71卷 199-213页

作者： Abdelhameed Ibrahim Hattan F.Abutarboush Ali Wagdy Mohamed Mohamad Fouad El-Sayed M.El-kenawy Computer Engineering and Control Systems Department Faculty of EngineeringMansoura UniversityMansoura35516Egypt Electrical Engineering Department College of EngineeringTaibah UniversityMedina42353Saudi Arabia Operations Research Department Faculty of Graduate Studies for Statistical ResearchCairo UniversityGiza12613Egypt Wireless Intelligent Networks Center(WINC) School of Engineering and Applied SciencesNile UniversityGizaEgypt Department of Communications and Electronics Delta Higher Institute of Engineering and TechnologyMansoura35111Egypt Faculty of Artificial Intelligence Delta University for Science and TechnologyEgypt

Metamaterial Antenna is a special class of antennas that uses metamaterial to enhance their *** size affects the quality factor and the radiation loss of the *** antennas can overcome the limitation of bandwidth for small *** learning(ML)model is recently applied to predict antenna *** can be used as an alternative approach to the trial-and-error process of finding proper parameters of the simulated *** accuracy of the prediction depends mainly on the selected *** models combine two or more base models to produce a better-enhanced *** this paper,a weighted average ensemble model is proposed to predict the bandwidth of the Metamaterial *** base models are used namely:Multilayer Perceptron(MLP)and Support Vector Machines(SVM).To calculate the weights for each model,an optimization algorithm is used to find the optimal weights of the *** Group-Based Cooperative Optimizer(DGCO)is employed to search for optimal weight for the base *** proposed model is compared with three based models and the average ensemble *** results show that the proposed model is better than other models and can predict antenna bandwidth efficiently.

关键词： Metamaterial antenna machine learning ensemble model multilayer perceptron support vector machines

Position: Understanding LLMs Requires More Than Statistical Generalization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Reizinger, Patrik Ujváry, Szilvia Mészáros, Anna Kerekes, Anna Brendel, Wieland Huszár, Ferenc Max Planck Institute for Intelligent Systems Tübingen Germany University of Cambridge United Kingdom AI Center UCL London United Kingdom Department of Computer Science ETH Zurich Switzerland Max Planck ETH Center for Learning Systems Switzerland ELLIS Institute Tübingen Germany Tübingen AI Center Germany

The last decade has seen blossoming research in deep learning theory attempting to answer, "Why does deep learning generalize?" A powerful shift in perspective precipitated this progress: the study of overparametrized models in the interpolation regime. In this paper, we argue that another perspective shift is due, since some of the desirable qualities of LLMs are not a consequence of good statistical generalization and require a separate theoretical explanation. Our core argument relies on the observation that AR probabilistic models are inherently non-identifiable: models zero or near-zero KL divergence apart—thus, equivalent test loss—can exhibit markedly different behaviors. We support our position with mathematical examples and empirical observations, illustrating why non-identifiability has practical relevance through three case studies: (1) the non-identifiability of zero-shot rule extrapolation;(2) the approximate non-identifiability of in-context learning;and (3) the non-identifiability of finetunability. We review promising research directions focusing on LLM-relevant generalization measures, transferability, and inductive biases. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep learning

An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Reizinger, Patrik Balestriero, Randall Klindt, David Brendel, Wieland Max Planck Institute for Intelligent Systems Tübingen Germany ELLIS Institute Tübingen Germany Department of Computer Science Brown University RI United States Cold Spring Harbor Laboratory Cold Spring HarborNY United States Tübingen AI Center Germany

Self-Supervised Learning (SSL) powers many current AI systems. As research interest and investment grow, the SSL design space continues to expand. The Platonic view of SSL, following the Platonic Representation Hypothesis (PRH), suggests that despite different methods and engineering approaches, all representations converge to the same Platonic ideal. However, this phenomenon lacks precise theoretical explanation. By synthesizing evidence from Identifiability Theory (IT), we show that the PRH can emerge in SSL. However, current IT cannot explain SSL’s empirical success. To bridge the gap between theory and practice, we propose expanding IT into what we term Singular Identifiability Theory (SITh), a broader theoretical framework encompassing the entire SSL pipeline. SITh would allow deeper insights into the implicit data assumptions in SSL and advance the field towards learning more interpretable and generalizable representations. We highlight three critical directions for future research: 1) training dynamics and convergence properties of SSL;2) the impact of finite samples, batch size, and data diversity;and 3) the role of inductive biases in architecture, augmentations, initialization schemes, and optimizers. © 2025, CC BY.

关键词： Engineering research

Large-Scale Radiomics Analysis for Prostate Cancer Detection Harnessing Machine and Deep Learning Models

学校读者我要写书评

暂无评论

Large-Scale Radiomics Analysis for Prostate Cancer Detection...

IEEE International Symposium on Biomedical Imaging

作者： Eugenia Mylona Dimitrios I. Zaridis Charalampos N. Kalantzopoulos Vasileios C. Pezoulas Nikolaos S. Tachos George K. Matsopoulos Daniele Regge Nikolaos Papanikolaou Manolis Tsiknakis Kostas Marias Dimitrios I. Fotiadis Biomedical Research Institute FORTH Ioannina Greece Unit of Medical Technology Intelligent Information Systems University of Ioannina Greece Biomedical Engineering Laboratory School of Electrical & Computer Engineering National Technical University of Athens Greece Institute of Computer Science FORTH Heraklion Greece Department of Radiology Candiolo Cancer Institute FPO-IRCCS Strada Provinciale Candiolo Italy Computational Clinical Imaging Group Champalimaud Foundation Portugal Computational Biomedicine Laboratory Institute of Computer Science FORTH Heraklion Greece Department of Electrical and Computer Engineering Hellenic Mediterranean University Heraklion Greece

ISBN: (数字)9798331520526

ISBN: (纸本)9798331520533

Prostate cancer (PCa) is the second most common cancer in men, making early detection crucial for effective treatment. This study explores the efficacy of machine learning (ML) and deep learning (DL) models in detecting PCa lesions using radiomics features derived from multiparametric MRI data. Leveraging a dataset of 7621 retrospective and 2132 prospective cases from 12 clinical centers across Europe, we compared the performance of various ML (LASSO, Random Forest, SVM, XGBoost) and DL models (CNN, MLP, Transformer, CNN&MLP). Radiomic features were extracted and refined through a robust feature selection process. Models' performance was evaluated through nested cross-validation (CV) and externally. The CNN&MLP model achieved the highest ROC AUC (0.86) and PR AUC (0.90) in CV, while LASSO showed consistent generalization in external validation. Our findings indicate that DL models excel in high-dimensional data analysis but face challenges in generalizing to external datasets, highlighting the importance of external validation for clinical applicability.

关键词： Deep learning Support vector machines Analytical models Biological system modeling Feature extraction Transformers Lesions Prostate cancer Radiomics Principal component analysis

Towards High-resolution 3D Anomaly Detection via Group-Level Feature Contrastive Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhu, Hongze Xie, Guoyang Hou, Chengbin Dai, Tao Gao, Can Wang, Jinbao Shen, Linlin National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen China Department of Computer Science City University of Hong Kong Hong Kong Department of Intelligent Manufacturing CATL Ningde China Fuzhou Fuyao Institute for Advanced Study Fuyao University of Science and Technology Fuzhou China College of Computer Science and Software Engineering Shenzhen University Shenzhen China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen China Shenzhen University Shenzhen China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China

High-resolution point clouds (HRPCD) anomaly detection (AD) plays a critical role in precision machining and high-end equipment manufacturing. Despite considerable 3D-AD methods that have been proposed recently, they still cannot meet the requirements of the HRPCD-AD task. There are several challenges: i) It is difficult to directly capture HRPCD information due to large amounts of points at the sample level;ii) The advanced transformer-based methods usually obtain anisotropic features, leading to degradation of the representation;iii) The proportion of abnormal areas is very small, which makes it difficult to characterize. To address these challenges, we propose a novel group-level feature-based network, called Group3AD, which has a significantly efficient representation ability. First, we design an Intercluster Uniformity Network (IUN) to present the mapping of different groups in the feature space as several clusters, and obtain a more uniform distribution between clusters representing different parts of the point clouds in the feature space. Then, an Intracluster Alignment Network (IAN) is designed to encourage groups within the cluster to be distributed tightly in the feature space. In addition, we propose an Adaptive Group-Center Selection (AGCS) based on geometric information to improve the pixel density of potential anomalous regions during inference. The experimental results verify the effectiveness of our proposed Group3AD, which surpasses Reg3D-AD by the margin of 5% in terms of object-level AUROC on Real3DAD. We provide the code and supplementary information on our website: https://***/M-3LAB/Group3AD. Copyright © 2024, The Authors. All rights reserved.

关键词： Anomaly detection

Efficient and High-quality Recommendations via Momentum-incorporated Parallel Stochastic Gradient Descent-Based Learning

学校读者我要写书评

暂无评论

IEEE/CAA Journal of Automatica Sinica 2021年第2期8卷 402-411页

作者： Xin Luo Wen Qin Ani Dong Khaled Sedraoui MengChu Zhou the School of Computer Science and Technology Dongguan University of TechnologyDongguan 523808 Hengrui(Chongqing)Artificial Intelligence Research Center Department of Big Data Analyses TechniquesCloudwalkChongqing 401331China the School of Computer Science and Technology Chongqing University of Posts and TelecommunicationsChongqing 400065 Chongqing Engineering Research Center of Big Data Application for Smart Cities Chongqing Institute of Green and Intelligent TechnologyChinese Academy of SciencesChongqing 400714China the Department of Computer and Information Science City College of Dongguan University of TechnologyDongguan 523419China the Department of Electrical and Computer Engineering Faculty of Engineeringand Center of Research Excellence in Renewable Energy and Power SystemsKing Abdulaziz UniversityJeddah 21481Saudi Arabia the Department of Electrical and Computer Engineering New Jersey Institute of TechnologyNewarkNJ 07102 USA the Center of Research Excellence in Renewable Energy and Power Systems King Abdulaziz UniversityJeddah 21481Saudi Arabia

A recommender system(RS)relying on latent factor analysis usually adopts stochastic gradient descent(SGD)as its learning ***,owing to its serial mechanism,an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial *** at addressing this issue,this study proposes a momentum-incorporated parallel stochastic gradient descent(MPSGD)algorithm,whose main idea is two-fold:a)implementing parallelization via a novel datasplitting strategy,and b)accelerating convergence rate by integrating momentum effects into its training *** it,an MPSGD-based latent factor(MLF)model is achieved,which is capable of performing efficient and high-quality *** results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm,an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.

关键词： Big data industrial application industrial data latent factor analysis machine learning parallel algorithm recommender system(RS) stochastic gradient descent(SGD)

Objective diagnosis of ADHD through movement analysis by using a smart chair with piezoelectric material

学校读者我要写书评

暂无评论

Pediatrics and Neonatology 2023年第1期64卷 46-52页

作者： Chang, Tung-Ming Wu, Rong-Ching Yang, Rei-Cheng Chiang, Ching-Tai Chiu, Yi-Hung Ouyang, Chen-Sen Wang, Yun-Ming Wu, Ming-Hsu Lin, Guang-Chung Lin, Lung-Chang Department of Pediatric Neurology Changhua Christian Children's Hospital Changhua Taiwan Department of Electrical Engineering I-Shou University Taiwan Departments of Pediatrics Kaohsiung Medical University Hospital Kaohsiung Medical University Taiwan Department of Computer and Communication National Pingtung University Taiwan Department of Information Engineering I-Shou University Taiwan Department of Biological Science and Technology Institute of Molecular Medicine and Bioengineering Center for Intelligent Drug Systems and Smart Bio-Devices (IDS2B) National Yang Ming Chiao Tung University Taiwan Department of Pediatrics School of Medicine College of Medicine Kaohsiung Medical University Taiwan Department of Biological Science and Technology College of Biological Science and Technology National Yang Ming Chiao Tung University Hsinchu Taiwan

Background: Attention-deficit/hyperactivity disorder (ADHD) is the most common neuropsychiatric disorder in schoolchildren. ADHD diagnoses are generally made based on criteria from the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. The diagnosis is made clinically based on observation and information provided by parents and teachers, which is highly subjective and can lead to disparate results. Considering that hyperactivity is one of the main symptoms of ADHD, the inaccuracy of ADHD diagnosis based on subjective criteria necessitates the identification of a method to objectively diagnose ADHD. Methods: In this study, a medical chair containing a piezoelectric material was applied to objectively analyze movements of patients with ADHD, which were compared with those of patients without ADHD. This study enrolled 62 patients—31 patients with ADHD and 31 patients without ADHD. During the clinical evaluation, participants’ movements were recorded by the piezoelectric material for analysis. The variance, zero-crossing rate, and high energy rate of movements were subsequently analyzed. Results: The results revealed that the variance, zero-crossing rate, and high energy rate were significantly higher in patients with ADHD than in those without ADHD. Classification performance was excellent in both groups, with the area under the curve as high as 98.00%. Conclusion: Our findings suggest that the use of a smart chair equipped with piezoelectric material is an objective and potentially useful method for supporting the diagnosis of ADHD. © 2022 Taiwan Pediatric Association

关键词： Attention-deficit/hyperactivity disorder piezoelectric material smart chair