In sub-second stream computing, the answer to a complex query usually depends on operations of aggregation or join on streams, especially multi-way theta join. Some attribute keys are not distributed uniformly, which ...
详细信息
ISBN:
(纸本)9781728190747
In sub-second stream computing, the answer to a complex query usually depends on operations of aggregation or join on streams, especially multi-way theta join. Some attribute keys are not distributed uniformly, which is called the data intrinsic skew problem, such as taxi car plate in GPS trajectories and transaction records, or stock code in stock quotes and investment portfolios etc. In this paper, we define the concept of key redundancy for single stream as the degree of data intrinsic skew, and joint key redundancy for multi-way streams. We present an execution model for multi-way stream theta joins with a fine-grained cost model to evaluate its performance. We propose a solution named Group Join (GroJoin) to make use of key redundancy during transmission and execution in a cluster. GroJoin is adaptive to data intrinsic skew in the way that it depends on the grouping condition we find out, i.e., the selectivity of theta join results should be smaller than 25%. Experiments are carried out by our MS-Generator to produce multi-way streams, and the simulation results show that GroJoin can decrease at most 45% transmission overheads with different key redundancies and value-key proportionality coefficients, and reduce at most 70% query delay with different key distributions. We further implement GroJoin in Multi-Way Stream Theta Join by Spark Streaming. The experimental results demonstrate that there are about 40%-50% join latency reduced after our optimization with a very small computation cost.
By integrating edge computing with parallelcomputing, distributed edge computing (DEC) makes use of distributed devices in edge networks to perform computing in parallel, which can substantially reduce service delays...
详细信息
ISBN:
(纸本)9781728164120
By integrating edge computing with parallelcomputing, distributed edge computing (DEC) makes use of distributed devices in edge networks to perform computing in parallel, which can substantially reduce service delays. In this paper, we explore DEC that exploits distributed edge devices connected by a wireless network to perform a computation task offloaded from an end device. In particular, we study the fundamental problem of minimizing the delay of executing a distributed algorithm of the computation task. We first establish some structural properties of the optimal communication scheduling policy. Then, given these properties, we characterize the optimal computation allocation policy, which can be found by an efficient algorithm. Next, based on the optimal computation allocation, we characterize the optimal scheduling order of communications for some special cases, and develop an efficient algorithm with a finite approximation ratio to find it for the general case. Last, based on the optimal computation allocation and communication scheduling, we further show that the optimal selection of devices can be found efficiently for some special cases. Our results provide some useful insights for the optimal computation-communication co-design. We evaluate the performance of the theoretical findings using simulations.
Reinforcement learning is used in many applications in artificial intelligence fields such as computer vision and environmental contextualized decision-making scenarios; virtual human swarms provide research direction...
详细信息
Reinforcement learning is used in many applications in artificial intelligence fields such as computer vision and environmental contextualized decision-making scenarios; virtual human swarms provide research directions for multi-intelligent body collaboration and environmental field detection in multi-intelligent body swarms. Football is a group sport, which is characterized by its holistic nature, confrontation, versatility, and ease of implementation, as well as the characteristics of both individual intelligence and group intelligence, and is a typical application scenario for multi-intelligence collaboration. This paper takes football as the research object to study the team collaboration problem and team gaming problem of the virtual human swarm in a specific environment and solves the cooperation and competition relationship between intelligence with reinforcement learning of multi-intelligence training.
The method for calculating the attitude of the projectile requires high real-time performance and a large amount of calculation. The parallel calculation method of the missile-borne signal is studied. For the linear i...
详细信息
ISBN:
(纸本)9781728180250
The method for calculating the attitude of the projectile requires high real-time performance and a large amount of calculation. The parallel calculation method of the missile-borne signal is studied. For the linear iterative equations in the attitude solution algorithm, a parallel calculation based on Newton method is proposed. This method decomposes linear iterative equations in parallel to form a distributed multi-core parallel solution process, which can effectively optimize the algorithm solution time. Experiments were conducted through four sets of data with different data lengths and frame rates to verify the execution efficiency of multi-core parallelcomputing.
Regression analysis and classification can be done using a supervised learning technique called Support Vector Machine (SVM) which is one of many such methods. The method creates hyperplanes which are used to analyze ...
详细信息
We introduce SpDISTAL, a compiler for sparse tensor algebra that targets distributed systems. SpDISTAL combines separate descriptions of tensor algebra expressions, sparse data structures, data distribution, and compu...
详细信息
We introduce SpDISTAL, a compiler for sparse tensor algebra that targets distributed systems. SpDISTAL combines separate descriptions of tensor algebra expressions, sparse data structures, data distribution, and computation distribution. Thus, it enables distributed execution of sparse tensor algebra expressions with a wide variety of sparse data structures and data distributions. SpDISTAL is implemented as a C++ library that targets a distributed task-based runtime system and can generate code for nodes with both multi-core CPUs and multiple GPUs. SpDISTAL generates distributed code that achieves performance competitive with hand-written distributed functions for specific sparse tensor algebra expressions and that outperforms general interpretation-based systems by one to two orders of magnitude.
Nowadays, side-information is widely used to rein-force the user-item interaction and helps to handle the sparsity issue and cold start problem of conventional recommendation algorithms. Due to the overlook of the rel...
详细信息
Nowadays, side-information is widely used to rein-force the user-item interaction and helps to handle the sparsity issue and cold start problem of conventional recommendation algorithms. Due to the overlook of the relationship between items and entities and the higher-order connectivity information, most existing approaches are hard to get users' deep preferences. In this paper, we propose KGANCL, a Knowledge-aware Graph Attention Network with distributed & Cross Learning. It focuses on using different forms of the knowledge graph to strengthen both users' and items' embedding representations, respectively. Firstly, Graph Attention Network is adopted for user embedding learning, which can give different importance scores to different neighbors, and a user-item KG graph is used to integrate adjacent information to enhance the representation. Secondly, a cross module is used for item embedding learning, which shares the high-order interaction between the recommender system and the knowledge graph. We also use the idea of distributed processing for embeddings in different entities to improve the learning efficiency. Experimental results demonstrate that KGANCL can provide better recommendations compared with the state-of-the-art baseline models. Our model can also maintain superior prediction accuracy even in little-known interaction scenarios.
Java Virtual Machine (JVM) is the fundamental software system that supports the interpretation and execution of Java bytecode. To support the surging performance demands for the increasingly complex and large-scale Ja...
Java Virtual Machine (JVM) is the fundamental software system that supports the interpretation and execution of Java bytecode. To support the surging performance demands for the increasingly complex and large-scale Java programs, Just-In-Time (JIT) compiler was proposed to perform sophisticated runtime optimization. However, this inevitably induces various bugs, which are becoming more pervasive over the decades and can often cause significant consequences. To facilitate the design of effective and efficient testing techniques to detect JIT compiler bugs. This study first performs a preliminary study aiming to understand the characteristics of JIT compiler bugs and the corresponding triggering test cases. Inspired by the empirical findings, we propose JOpFuzzer, a new JVM testing approach with a specific focus on JIT compiler bugs. The main novelty of JOpFuzzer is embodied in three aspects. First, besides generating new seeds, JOpFuzzer also searches for diverse configurations along the new dimension of optimization options. Second, JOpFuzzer learns the correlations between various code features and different optimization options to guide the process of seed mutation and option exploration. Third, it leverages the profile data, which can reveal the program execution information, to guide the fuzzing process. Such nov-elties enable JOpFuzzer to effectively and efficiently explore the two-dimensional input spaces. Extensive evaluation shows that JOpFuzzer outperforms the state-of-the-art approaches in terms of the achieved code coverages. More importantly, it has detected 41 bugs in OpenJDK, and 25 of them have already been confirmed or fixed by the corresponding developers.
distributed energy resources (DERs) are attractive because of their flexibility and demand response capabilities;however, there are numerous challenges concerning their integration into the electric grid - including l...
详细信息
ISBN:
(纸本)9781728161273
distributed energy resources (DERs) are attractive because of their flexibility and demand response capabilities;however, there are numerous challenges concerning their integration into the electric grid - including lack of visibility and control, as well as misalignments in the interests of privately-owned DERs with respect to the collective interests of the grid. In order to coordinate the behavior of these DERs, we treat the grid as a multiagent system and propose a service-oriented broker architecture (SOBA). SOBA enables the behavior of privately owned DERs to be influenced by system operators through service requests, and autonomous peer discovery. We illustrate SOBA's features and motivate service requests through a scenario with a network of small-scale solar photovoltaics, inverters, and batteries.
Executing complicated computations in parallel increases the speed of computing and brings user delight to the system. Decomposing the program into several small programs and running multiple parallel processors are m...
详细信息
ISBN:
(纸本)9781728110516
Executing complicated computations in parallel increases the speed of computing and brings user delight to the system. Decomposing the program into several small programs and running multiple parallel processors are modeled by Directed Acyclic Graph. Scheduling nodes to execute this task graph is an important problem that will speed up computations. Since task scheduling in this graph belongs to NP-hard problems, various algorithms were developed for node scheduling to contribute to quality service delivery. The present study brought a heuristic algorithm named looking ahead sequencing algorithm (LASA) to cope with static scheduling in heterogeneous distributedcomputing systems with the intention of minimizing the schedule length of the user application. In the algorithm proposed here, looking ahead is considered as a criterion for prioritizing tasks. Also, a property called Emphasized Processor has been added to the algorithm to emphasize the task execution on a particular processor. The effectiveness of the algorithm was shown on few workflow type applications and the results of the algorithm implementation were compared with two more heuristic and meta-heuristic algorithms.
暂无评论