检索结果-内蒙古大学图书馆

Scalable, reproducible, and cost-effective processing of large-scale medical imaging datasets

学校读者我要写书评

暂无评论

Scalable, reproducible, and cost-effective processing of lar...

Medical Imaging 2025: Imaging Informatics

作者： Kim, Michael E. Ramadass, Karthik Gao, Chenyu Kanakaraj, Praitayini Newlin, Nancy R. Rudravaram, Gaurav Schilling, Kurt G. Dewey, Blake E. Archer, Derek Hohman, Timothy J. Li, Zhiyuan Bao, Shunxing Landman, Bennett A. Khairi, Nazirah Mohd Vanderbilt University Department of Computer Science NashvilleTN United States Vanderbilt University Department of Electrical and Computer Engineering NashvilleTN United States Vanderbilt University Medical Center Department of Radiology and Radiological Sciences NashvilleTN United States Department of Neurology Johns Hopkins University School of Medicine BaltimoreMD United States Vanderbilt University Medical Center Vanderbilt Memory and Alzheimer's Center NashvilleTN United States Vanderbilt University Medical Center Vanderbilt Genetics Institute NashvilleTN United States Vanderbilt University Department of Biomedical Engineering NashvilleTN United States Vanderbilt University Institute of Imaging Science NashvilleTN United States

ISBN: (纸本)9781510686007

Curating, processing, and combining large-scale medical imaging datasets from national studies is a non-trivial task due to the intense computation and data throughput required, variability of acquired data, and associated financial overhead. Existing platforms or tools for large-scale data curation, processing, and storage have difficulty achieving a viable cost-to-scale ratio of computation speed for research purposes, either being too slow or too expensive. Additionally, management and consistency of processing large data in a team-driven manner is a non-trivial task. We design a BIDS-compliant method for an efficient and robust data processing pipeline of large-scale diffusion-weighted and T1-weighted MRI data compatible with low-cost, high-efficiency computing systems. Our method accomplishes automated querying of data available for processing and process running in a consistent and reproducible manner that has long-term stability, while using heterogenous low-cost computational resources and storage systems for efficient processing and data transfer. We demonstrate how our organizational structure permits efficiency in a semi-automated data processing pipeline and show how our method is comparable in processing time to cloud-based computation while being almost 20 times more cost-effective. Our design allows for fast data throughput speeds and low latency to reduce the time for data transfer between storage servers and computation servers, achieving an average of 0.60 Gb/s compared to 0.33 Gb/s for using cloud-based processing methods. The design of our workflow engine permits quick process running while maintaining flexibility to adapt to newly acquired data. © 2025 SPIE

关键词： batch data processing

SuperDecode: A versatile toolkit for mutation analysis in genome editing

学校读者我要写书评

暂无评论

aBIOTECH 2025年 1-4页

作者： Hua, Guoying He, Chenfei Zuo, Erwei Shenzhen Branch Guangdong Laboratory for Lingnan Modern Agriculture Agricultural Genomics Institute at Shenzhen Chinese Academy of Agricultural Sciences Shenzhen518120 China

The CRISPR-Cas system has revolutionized modern life sciences, enabling groundbreaking applications ranging from functional genomics to therapeutic development. Despite its transformative potential, significant technical limitations persist in current computational tools for quantifying editing efficiency - particularly concerning data processing capabilities, analytical throughput, and operational flexibility. This research presents SuperDecode, a novel computational framework designed to address these methodological constraints. The SuperDecode offers key advantages, including local processing capabilities, large-size sequencing files, batch-processing, and diversified operational functions. © Agricultural Information Institute, Chinese Academy of Agricultural Sciences 2025.

关键词： batch data processing

Automated Precision in Food Packaging: A Cost-Effective Solution for Emerging Companies 4th

学校读者我要写书评

暂无评论

Automated Precision in Food Packaging: A Cost-Effective Sol...

4th International Conference on Advanced Research in Technologies, Information, Innovation and Sustainability 2024, ARTIIS 2024

作者： Terneus, Juan Diego Moya, Viviana Abedrabbo, Faruk Vásconez, Juan Pablo Moya, Marcelo Facultad de Ciencias Técnicas Universidad Internacional Del Ecuador UIDE Quito170411 Ecuador Energy Transformation Center Faculty of Engineering Universidad Andres Bello Santiago7500971 Chile

ISBN: (纸本)9783031832093

This study aims to automate the packaging process efficiently and enhance production in emerging companies. A prototype that integrates mechanical, electronic, and control design was developed to achieve mass-controlled filling with precise measurements and efficient batch processing. The system features a screw conveyor mechanism and a rotary drum mechanism, employing a cascade control strategy with a Proportional-Integral (PI) controller to accurately regulate motor speed. Extensive testing of the filling machine demonstrated its capability to fill 100 g containers with an error margin of ±5 g and an average filling time of 1.08 min per container. These results highlighted the system’s exceptional accuracy and efficiency. A cost analysis of implementing this machine in the Ecuadorian Tervide Company was also provided. The proposed methodology shows significant potential to enhance efficiency and accuracy in filling food products for emerging companies, establishing a precedent for sustainable production improvements globally. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： batch data processing

FastER: Fast On-Demand Entity Resolution in Property Graphs

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Shujing Kwashie, Selasi Bewong, Michael Hu, Junwei Nofong, Vincent M. Miao, Shiqi Feng, Zaiwen Huazhong Agricultural University Wuhan China AI & Cyber Futures Institute Charles Sturt University Australia School of Computing Mathematics and Engineering Charles Sturt University Australia Department of Computer Science and Engineering University of Mines and Technology Ghana

Entity resolution (ER) is the problem of identifying and linking database records that refer to the same real-world entity. Traditional ER methods use batch processing, which becomes impractical with growing data volumes due to high computational costs and lack of real-time capabilities. In many applications, users need to resolve entities for only a small portion of their data, making full data processing unnecessary—a scenario known as "ER-on-demand". This paper proposes FastER, an efficient ER-on-demand framework for property graphs. Our approach uses graph differential dependencies (GDDs) as a knowledge encoding language to design effective filtering mechanisms that leverage both structural and attribute semantics of graphs. We construct a blocking graph from filtered subgraphs to reduce the number of candidate entity pairs requiring comparison. Additionally, FastER incorporates Progressive Profile Scheduling (PPS), allowing the system to incrementally produce results throughout the resolution process. Extensive evaluations on multiple benchmark datasets demonstrate that FastER significantly outperforms state-of-the-art ER methods in computational efficiency and real-time processing for on-demand tasks while ensuring reliability. We make FastER publicly available at the Github link here. © 2025, CC BY-NC-SA.

关键词： batch data processing

Serverless Approach to Running Resource-Intensive STAR Aligner

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Kica, Piotr Orzechowski, Michal Malawski, Maciej Sano Centre for Computational Medicine Kraków Poland Faculty of Computer Science AGH University of Kraków Poland Academic Computer Centre Cyfronet AGH Kraków Poland

The application of serverless computing for alignment of RNA-sequences can improve many existing bioinformatics workflows by reducing operational costs and execution times. This work analyzes the applicability of serverless services for running the STAR aligner, which is known for its accuracy and large memory requirement. This presents a challenge, as serverless services were designed for light and short tasks. Nevertheless, we successfully deploy a STAR-based pipeline on AWS ECS service, propose multiple optimizations, and perform experiment with 17 TBs of data. Results are compared against standard virtual machine (VM) based solution showing that serverless is a valid alternative for small-scale batch processing. However, in large-scale where efficiency matters the most, VMs are still recommended. © 2025, CC BY.

关键词： batch data processing

batch process quality prediction based on TFCMixer modeling

学校读者我要写书评

暂无评论

SSRN

SSRN 2025年

作者： Zhao, Xiaoqiang Liu, Yongyong Hui, Yonyong Liu, Kai Yao, Qinglei College of Electrical and Information Engineering Lanzhou University of Technology Lanzhou730050 China Gansu Key Laboratory Advanced Control for Industrial Processes Lanzhou University of Technology Lanzhou730050 China National Experimental Teaching Center of Electrical and Control Engineering Lanzhou University of Technology Lanzhou730050 China

Deep learning models are now widely used for the prediction of quality variables in batch processes. However, the dynamics and nonlinearities of batch processes, and long training time of deep learning models are unfavorable for quality prediction. To address these issues, an batch process quality prediction method based on the Temporal Feature Convolutional Mixer (TFCMixer) model is proposed. First, the maximum information coefficient (MIC) method is utilized to filter out the process variables that are more relevant to the quality variables to reduce the model training time. Second, a temporal feature convolution processor is designed to extract key information from the time series. On the one hand, the pro cessor processes the data in time dimension through convolution operation to learn the dynamic changes of the data in time, and utilizes double convolution operation to process the feature dimension in the data, so that the model can learn the interaction information between the features, which helps to deeply understand the complex relationship between the features and improve the prediction accuracy of the model;on the other hand, it incorporates a residual scale block to regulate the model's training time. Scale is incorporated to regulate the effect of residual signals in the final output, so as to adjust the performance of the model according to different tasks and data characteristics, and the Hard Swish activation function with continuous differentiability is introduced to stabilize the training of the model. Then, the temporal projection is combined with a temporal feature convolution processor to perform the conversion between time steps. Finally, experiments are conducted using data from the penicillin fermentation process and semiconductor wafer etching process. The results demonstrate the accuracy and effectiveness of the proposed model, which requires fewer iterations for training to achieve excellent prediction results. © 2025, The Authors. All ri

关键词： batch data processing

batch normalization does not improve initialization

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Dannemann, Joris Junike, Gero Carl von Ossietzky Universität Institut für Mathematik Oldenburg26129 Germany

batch normalization is one of the most important regularization techniques for neural networks, significantly improving training by centering the layers of the neural network. There have been several attempts to provide a theoretical justification for batch normalization. Santurkar and Tsipras (2018) [How does batch normalization help optimization? Advances in neural information processing systems, 31] claim that batch normalization improves initialization. We provide a counterexample showing that this claim is not true, i.e., batch normalization does not improve initialization. © 2025, CC BY.

关键词： batch data processing

SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Fahao Li, Peng Luan, Tom H. Su, Zhou Deng, Jing School of Computer Science and Engineering The University of Aizu Japan School of Cyber Science and Engineering Xi’an Jiaotong University China Department of Computer Science University of North Carolina Greensboro United States

Speculative decoding has been shown as an effective way to accelerate Large Language Model (LLM) inference by using a Small Speculative Model (SSM) to generate candidate tokens in a so-called speculation phase, which are subsequently verified by the LLM in a verification phase. However, current state-of-the-art speculative decoding approaches have three key limitations: handling requests with varying difficulty using homogeneous SSMs, lack of robust support for batch processing, and insufficient holistic optimization for both speculation and verification phases. In this paper, we introduce SPIN, an efficient LLM inference serving system based on speculative decoding, designed to address these challenges through three main innovations. First, SPIN improves token speculation by using multiple heterogeneous SSMs, with a learning-based algorithm for SSM selection that operates without prior knowledge of request difficulty. Second, SPIN employs a request decomposition method to minimize batching overhead during LLM verification. Finally, SPIN orchestrates speculation and verification phases by pipelining their executions on GPUs to achieve further acceleration. Experimental results demonstrate that SPIN significantly outperforms state-of-the-art methods, achieving a performance increase of approximately 2.28×. © 2025, CC BY.

关键词： batch data processing

Charm: The Missing Piece in ViT fine-tuning for Image Aesthetic Assessment

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Behrad, Fatemeh Tuytelaars, Tinne Wagemans, Johan KU Leuven University Belgium

The capacity of Vision transformers (ViTs) to handle variable-sized inputs is often constrained by computational complexity and batch processing limitations. Consequently, ViTs are typically trained on small, fixed-size images obtained through downscaling or cropping. While reducing computational burden, these methods result in significant information loss, negatively affecting tasks like image aesthetic assessment. We introduce Charm, a novel tokenization approach that preserves Composition, High-resolution, Aspect Ratio, and Multi-scale information simultaneously. Charm prioritizes high-resolution details in specific regions while downscaling others, enabling shorter fixed-size input sequences for ViTs while incorporating essential information. Charm is designed to be compatible with pre-trained ViTs and their learned positional embeddings. By providing multiscale input and introducing variety to input tokens, Charm improves ViT performance and generalizability for image aesthetic assessment. We avoid cropping or changing the aspect ratio to further preserve information. Extensive experiments demonstrate significant performance improvements on various image aesthetic and quality assessment datasets (up to 8.1 %) using a lightweight ViT backbone. Code and pre-trained models are available at GitHub. © 2025, CC BY.

关键词： batch data processing