Scaling Bayesian Network Parameter Learning With Expectation Maximization Using Mapreduce, Erik B. Reed, Ole J. Mengshoel
Nov 2012
Scaling Bayesian Network Parameter Learning With Expectation Maximization Using Mapreduce, Erik B. Reed, Ole J. Mengshoel
Ole J Mengshoel
Bayesian network (BN) parameter learning from incomplete data can be a computationally expensive task for incomplete data. Applying the EM algorithm to learn BN parameters is unfortunately susceptible to local optima and prone to premature convergence. We develop and experiment with two methods for improving EM parameter learning by using MapReduce: Age-Layered Expectation Maximization (ALEM) and Multiple Expectation Maximization (MEM). Leveraging MapReduce for distributed machine learning, these algorithms (i) operate on a (potentially large) population of BNs and (ii) partition the data set as is traditionally done with MapReduce machine learning. For example, we achieved gains using the Hadoop implementation …
Mapreduce For Bayesian Network Parameter Learning Using The Em Algorithm, Aniruddha Basak, Irina Brinster, Ole J. Mengshoel
Nov 2012
Mapreduce For Bayesian Network Parameter Learning Using The Em Algorithm, Aniruddha Basak, Irina Brinster, Ole J. Mengshoel
Ole J Mengshoel
This work applies the distributed computing framework MapReduce to Bayesian network parameter learning from incomplete data. We formulate the classical Expectation Maximization (EM) algorithm within the MapReduce framework. Analytically and experimentally we analyze the speed-up that can be obtained by means of MapReduce. We present details of the MapReduce formulation of EM, report speed-ups versus the sequential case, and carefully compare various Hadoop cluster configurations in experiments with Bayesian networks of different sizes and structures.
Software And System Health Management For Autonomous Robotics Missions, Johann Schumann, Timmy Mbaya, Ole J. Mengshoel
Sep 2012
Software And System Health Management For Autonomous Robotics Missions, Johann Schumann, Timmy Mbaya, Ole J. Mengshoel
Ole J Mengshoel
Advanced autonomous robotics space missions rely heavily on the flawless interaction of complex hardware, multiple sensors, and a mission-critical software system. This software system consists of an operating system, device drivers, controllers, and executives; recently highly complex AI-based autonomy software have also been introduced. Prior to launch, this software has to undergo rigorous verification and validation (V&V). Nevertheless, dormant software bugs, failing sensors, unexpected hardware-software interactions, and unanticipated environmental conditions—likely on a space exploration mission—can cause major software faults that can endanger the entire mission.
Our Integrated Software Health Management (ISWHM) system continuously monitors the hardware sensors and the software …
Reactive Bayesian Network Computation Using Feedback Control: An Empirical Study, Ole J. Mengshoel, Abe Ishihara, Erik Reed
Aug 2012
Reactive Bayesian Network Computation Using Feedback Control: An Empirical Study, Ole J. Mengshoel, Abe Ishihara, Erik Reed
Ole J Mengshoel
This paper investigates the challenge of integrating intelligent systems into varying computational platforms and application mixes while providing reactive (or soft real-time) response. We integrate Bayesian network computation with feedback control, thereby achieving our reactive objective. As a case study we investigate fault diagnosis using Bayesian networks. While we consider the likelihood weighting and junction tree propagation Bayesian network inference algorithms in some detail, we hypothesize that the techniques developed can be broadly applied to achieve reactive intelligent systems. In the empirical study of this paper we demonstrate reactive fault diagnosis for an electrical power system.
Accelerating Bayesian Network Parameter Learning Using Hadoop And Mapreduce, Aniruddha Basak, Irina Brinster, Xianheng Ma, Ole J. Mengshoel
Aug 2012
Accelerating Bayesian Network Parameter Learning Using Hadoop And Mapreduce, Aniruddha Basak, Irina Brinster, Xianheng Ma, Ole J. Mengshoel
Ole J Mengshoel
Learning conditional probability tables of large Bayesian Networks (BNs) with hidden nodes using the Expectation Maximization algorithm is heavily computationally intensive. There are at least two bottlenecks, namely the potentially huge data set size and the requirement for computation and memory resources. This work applies the distributed computing framework MapReduce to Bayesian parameter learning from complete and incomplete data. We formulate both traditional parameter learning (complete data) and the classical Expectation Maximization algorithm (incomplete data) within the MapReduce framework. Analytically and experimentally we analyze the speed-up that can be obtained by means of MapReduce. We present the details of our …
Adaptive Control Of Bayesian Network Computation, Erik Reed, Abe Ishihara, Ole J. Mengshoel
Jul 2012
Adaptive Control Of Bayesian Network Computation, Erik Reed, Abe Ishihara, Ole J. Mengshoel
Ole J Mengshoel
This paper considers the problem of providing, for computational processes, soft real-time (or reactive) response without the use of a hard real-time operating system. In particular, we focus on the problem of reactively computing fault diagnosis by means of different Bayesian network inference algorithms on non-real-time operating systems where low-criticality (background) process activity and system load is unpredictable.
To address this problem, we take in this paper a reconfigurable adaptive control approach. Computation time is modeled using an ARX model where the input consists of the maximum number of background processes allowed to run at any given time. To ensure …
Age-Layered Expectation Maximization For Parameter Learning In Bayesian Networks, Avneesh Saluja, Priya Sundararajan, Ole J. Mengshoel
Apr 2012
Age-Layered Expectation Maximization For Parameter Learning In Bayesian Networks, Avneesh Saluja, Priya Sundararajan, Ole J. Mengshoel
Ole J Mengshoel
The expectation maximization (EM) algorithm is a popular algorithm for parameter estimation in models with hidden variables. However, the algorithm has several non-trivial limitations, a significant one being variation in eventual solutions found, due to convergence to local optima. Several techniques have been proposed to allay this problem, for example initializing EM from multiple random starting points and selecting the highest likelihood out of all runs. In this work, we a) show that this method can be very expensive computationally for difficult Bayesian networks, and b) in response we propose an age-layered EM approach (ALEM) that efficiently discards less promising …
