检索结果-内蒙古大学图书馆

Variable Selection for Distributed Sparse Regression Under memory constraints

Communications in Mathematics and Statistics 2024年第2期12卷 307-338页

作者： Haofeng Wang Xuejun Jiang Min Zhou Jiancheng Jiang Department of Mathematics Harbin Institute of TechnologyHarbinPeople's Republic of China Department of Statistics and Data Science Southerm Univerity of Science and TechnologyShenzhenPeople's Republic of China Beijing Normal University-Hong Kong Baptist University United International College ZhuhaiPeople's Republic of China Department of Mathematics and Statistics University of North Carolina at CharlotteCharlotteUSA

This paper studies variable selection using the penalized likelihood method for dis-tributed sparse regression with large sample size n under a limited memory *** is a much needed research problem to be solved in the big data era.A naive divide-and-conquer method solving this problem is to split the whole data into N parts and run each part on one of N machines,aggregate the results from all machines via averaging,andﬁnally obtain the selected ***,it tends to select more noise variables,and the false discovery rate may not be well *** improve it by a special designed weighted average in *** the alternating direction method of multiplier can be used to deal with massive data in the literature,our proposed method reduces the computational burden a lot and performs better by mean square error in most ***,we establish asymptotic properties of the resulting estimators for the likelihood models with a diverging number of *** some regularity conditions,we establish oracle properties in the sense that our distributed estimator shares the same asymptotic efﬁciency as the estimator based on the full ***,a distributed penalized likelihood algorithm is proposed to reﬁne the results in the context of general ***,the proposed method is evaluated by simulations and a real example.

关键词： Variable selection Distributed sparse regression memory constraints Distributed penalized likelihood algorithm

来源：评论

学校读者我要写书评

暂无评论

Dealing with memory constraints in a non-dedicated linux cluster

引用

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS 2003年第1期17卷 39-48页

作者： Giné, F Solsona, F Hernández, P Luque, E Univ Lleida Lleida Spain Univ Autonoma Barcelona E-08193 Barcelona Spain

Our research is focused on keeping both local and parallel jobs together in a non-dedicated cluster and scheduling them efficiently. In such a system, memory becomes a critical resource for both kinds of job. Thus, the minimization of the impact caused by the overflow of physical into virtual memory on the performance of distributed jobs together with the development of an efficient memory management adapted to the needs of NOWs (Networks of Workstations) are major issues in this kind of environment. An algorithm is presented to adjust time slices dynamically to the necessities of distributed and local tasks in order to reduce the number of page faults across the cluster. Additionally, a memory management policy aimed at preserving enough memory for both work loads is evaluated. Our proposals are implemented in a Linux cluster and compared with alternative algorithms.

关键词： cluster computing scheduling memory constraints

来源：评论

学校读者我要写书评

暂无评论

Toward Bounds on Parallel Execution Times of Task Graphs on Multicores With memory constraints

引用

IEEE ACCESS 2019年 7卷 52778-52789页

作者： Song, Jiangong Li, Qinyong Ma, Shilong Beihang Univ State Key Lab Software Dev Environm Beijing 100083 Peoples R China

As more and more parallel programs are migrating to shared computing platforms, bounding their parallel execution times under resource constraints is particularly important for their efficient executions. The parallel program is often modeled as a task graph, which is composed of a collection of control or data-dependent sub-tasks organized as a directed acyclic graph (DAG) for scheduling. In this paper, we propose a simple yet effective method to bound the parallel execution time of a task graph when the memory resources are constrained. The essence of this method is to extend Brent's theorem by incorporating the memory factor. To this end, we introduce a concept of range of concurrent tasks (RCT) and leverage it to estimate an upper bound on the parallel execution time of task graph with respect to work-conserving scheduling algorithm. And also we exploit the estimated bound to develop a metric to help determine optimal memory capacity for a given task graph. Through an empirical study, we evaluate how good the estimated bound is via a designed scheduling algorithm and demonstrate the effectiveness of the metric in the selection of optimal memory capacity for a task graph.

关键词： Task graph scheduling time bounds parallel execution multicores shared memory memory constraints

来源：评论

学校读者我要写书评

暂无评论

Mondrian forest for data stream classification under memory constraints

引用

DATA MINING AND KNOWLEDGE DISCOVERY 2024年第2期38卷 569-596页

作者： Khannouz, Martin Glatard, Tristan Concordia Univ Dept Comp Sci & Software Engn Montreal PQ Canada

Supervised learning algorithms generally assume the availability of enough memory to store data models during the training and test phases. However, this assumption is unrealistic when data comes in the form of infinite data streams, or when learning algorithms are deployed on devices with reduced amounts of memory. In this paper, we adapt the online Mondrian forest classification algorithm to work with memory constraints on data streams. In particular, we design five out-of-memory strategies to update Mondrian trees with new data points when the memory limit is reached. Moreover, we design node trimming mechanisms to make Mondrian trees more robust to concept drifts under memory constraints. We evaluate our algorithms on a variety of real and simulated datasets, and we conclude with recommendations on their use in different situations: the Extend Node strategy appears as the best out-of-memory strategy in all configurations, whereas different node trimming mechanisms should be adopted depending on whether a concept drift is expected. All our methods are implemented in the OrpailleCC open-source library and are ready to be used on embedded systems and connected objects.

关键词： Mondrian forest Trimming Concept drift Data stream memory constraints

来源：评论

学校读者我要写书评

暂无评论

OPTIMAL LIST ORDER UNDER PARTIAL memory constraints

引用

JOURNAL OF APPLIED PROBABILITY 1980年第4期17卷 1004-1015页

作者： KAN, YC ROSS, SM University of California Berkeley

Suppose that we are given a set of n elements which are to be arranged in some order. At each unit of time a request is made to retrieve one of these elements — the ith being requested with probability Pi. We show that the rule which always moves the requested element one closer to the front of the line minimizes the average position of the element requested among a wide class of rules for all probability vectors of the form P1 = p, P2= · ·· = Pn = (1 – p)/(n − 1). We also consider the above problem when the decision-maker is allowed to utilize such rules as ‘only make a change if the same element has been requested k times in a row', and show that as k approaches infinity we can do as well as if we knew the values of the Pi.

关键词： OPTIMAL LIST ORDER memory constraints TRANSPOSITION RULE k-IN-A-ROW RANDOMIZATION TIME REVERSIBILITY

来源：评论

学校读者我要写书评

暂无评论

LWMPI: An MPI library for NoC-based lightweight manycore processors with on-chip memory constraints

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2023年第17期35卷

作者： Uller, Joao Fellipe Souto, Joao Vicente Penna, Pedro Henrique Castro, Marcio Freitas, Henrique Mehaut, Jean-Francois Univ Fed Santa Catarina UFSC Lab Pesquisa Sistemas Distribuidos LaPeSD Florianopolis SC Brazil Pontificia Univ Catolica Minas Gerais PUC Minas Comp Architecture & Parallel Proc Team CArT Belo Horizonte MG Brazil Univ Grenoble Alpes UGA Lab Informat Grenoble LIG St Martin Dheres France

Lightweight manycore processors deliver high performance and energy efficiency by bundling hundreds of low-power cores, a distributed memory architecture with small local memories and Networks-on-Chip in a single die. However, the lack of rich and portable programming models for these processors makes software development a challenging task. Currently, two approaches are employed to address programmability in lightweight manycores: Operating Systems (OSes) and baremetal runtime libraries. The former provides portability but exposes complex Operating System (OS)-level programming interfaces to developers. The latter focuses on providing rich and high performance interfaces, which are vendor-specific and yield to non-portable software. In this work, we address these programmability and portability challenges by combining a rich OS with a well-known standard for parallel programming. We propose a portable and lightweight Message Passing Interface (MPI) library (LWMPI) designed from scratch to cope with restrictions and intricacies of lightweight manycores. We integrated LWMPI into Nanvix, an open-source distributed OS that runs on silicon lightweight manycores. The results obtained with a synthetic benchmark and a subset of the CAP Bench applications running on Kalray MPPA-256 unveil that LWMPI not only delivers a lightweight and richer programming interface but also presents good performance and scalability results.

关键词： lightweight manycores memory constraints MPI runtime systems

来源：评论

学校读者我要写书评

暂无评论

ASYMPTOTICALLY OPTIMAL HYPOTHESIS-TESTING WITH memory constraints

引用

ANNALS OF STATISTICS 1991年第2期19卷 982-998页

作者： BUCKLEW, JA NEY, PE UNIV WISCONSIN DEPT MATHMADISONWI 53706

The binary hypothesis testing problem of deciding between two Markov chains is formulated under memory constraints. The optimality criterion used is the exponential rate with which the probability of error approaches zero as the sample size tends to infinity. The optimal memory constrained test is shown to be the solution of a set of equations derived from suitable large deviation twistings of the transition matrices under the two hypotheses. A computational algorithm and some examples are given.

关键词： TESTING HYPOTHESES memory constraints LARGE DEVIATIONS

来源：评论

学校读者我要写书评

暂无评论

A memory-Constraint-Aware List Scheduling Algorithm for memory-Constraint Heterogeneous Muti-Processor System

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2023年第4期34卷 1082-1099页

作者： Yao, Yu Song, Yukun Huang, Ying Ni, Wei Zhang, Duoli Hefei Univ Technol Sch Microelect Hefei Anhui 230601 Peoples R China

An effective scheduling algorithm is vital for the execution efficiency of applications on Heterogeneous Muti-Processor System (HMPS), especially memory-Constraint Heterogeneous Muti-Processor System (MCHMPS). Stringent local and external memory constraints have significant impact on the execution performance of applications executed on MCHMPS, predictability is also a critical factor for task scheduling on MCHMPS. Therefore, a novel list scheduling algorithm termed memory-constraint-aware Improved Predict Priority and Optimistic Processor Selection Scheduling (MIPPOSS), essentially a heuristic search optimization algorithm, is proposed in this paper. In MIPPOSS, a predictive approach is applied for task prioritization and processor selection, and a novel memory-constraint-aware approach is employed in the processor selection phase. MIPPOSS has polynomial complexity and produces better results for application scheduling on target architecture. Randomly generated DAGs and 3 real-world applications experiments, including Cybershake, LIGO, and Montage, show that MIPPOSS outperforms the other five competing algorithms by a large margin.

关键词： Task analysis Scheduling memory management Costs Computational modeling Scheduling algorithms Time complexity Heterogeneous systems memory constraints parallel computing list scheduling

来源：评论

学校读者我要写书评

暂无评论

Cascaded multimodal biometric recognition framework

引用

IET BIOMETRICS 2014年第1期3卷 16-28页

作者： Baig, Asim Bouridane, Ahmed Kurugollu, Fatih Albesher, Badr Queens Univ Belfast Inst Elect Commun & Informat Technol Belfast BT3 9DT Antrim North Ireland Muhammad Ali Jinnah Univ Dept Elect & Comp Engn Islamabad Pakistan Northumbria Univ Dept Comp Sci & Digital Technol Newcastle Upon Tyne NE12 1XE Tyne & Wear England Imam Mohammad Ibn Saud Islamic Univ Riyadh 11439 Saudi Arabia

A practically viable multi-biometric recognition system should not only be stable, robust and accurate but should also adhere to real-time processing speed and memory constraints. This study proposes a cascaded classifier-based framework for use in biometric recognition systems. The proposed framework utilises a set of weak classifiers to reduce the enrolled users' dataset to a small list of candidate users. This list is then used by a strong classifier set as the final stage of the cascade to formulate the decision. At each stage, the candidate list is generated by a Mahalanobis distance-based match score quality measure. One of the key features of the authors framework is that each classifier in the ensemble can be designed to use a different modality thus providing the advantages of a truly multimodal biometric recognition system. In addition, it is one of the first truly multimodal cascaded classifier-based approaches for biometric recognition. The performance of the proposed system is evaluated both for single and multimodalities to demonstrate the effectiveness of the approach.

关键词： multimodal cascaded classifier-based approaches user dataset biometrics (access control) image recognition cascaded multimodal biometric recognition framework real-time processing speed Mahalanobis distance-based match score quality measure memory constraints Computer vision and image processing techniques image classification Image recognition viable multibiometric recognition system

来源：评论

学校读者我要写书评

暂无评论

An efficient Huffman table sharing method for memory-constrained entropy coding of multiple sources

引用

SIGNAL PROCESSING-IMAGE COMMUNICATION 1998年第2期13卷 99-110页

作者： Lee, SJ Lee, CW SK Telecom Cent R&D Ctr Core Technol Dev Lab Taejon 305348 South Korea Seoul Natl Univ Sch Elect Engn Inst New Media & Commun Seoul 151 South Korea

For optimal entropy coding of multiple sources, the encoder and the decoder have to retain a separate Huffman table for each source. In many practical cases, however, available memory is usually restricted and therefore it is necessary for some sources to share a Huffman table. Recently, we developed an iterative algorithm, which leads to locally optimal sharing of Huffman tables (Lee et al., 1995). In this paper, we examine the iterative algorithm in detail and present some modification methods which improve the sharing performance and reduce computational complexity. First, considering that the iterative algorithm provides only locally optimum, pie introduce an unused table processing method and two initialization methods, splitting and merging, for the iterative algorithm.,And fixed-length coding is introduced for the encoding of some sources, which provides considerable performance improvement in case only a small number of Huffman tables are used. In addition, we present a simple approximation method of codelengths, which reduces the computational complexity of the sharing algorithm with little performance degradation. Simulations show that performance can be further improved with these modifications. For practical applications, we also present a search method for fast determining the number of Huffman tables and the Huffman table size, which, under a given memory constraint, minimize average bit-rate. The proposed sharing method can be widely applied to many high-order entropy coding systems with memory constraints. (C) 1998 Elsevier Science B.V. All rights reserved.

关键词： Huffman table sharing iterative algorithm unused Huffman table composite source splitting merging codelength approximation memory constraints

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：