Physical Sciences and Mathematics Commons^™

High Performance Computing Markov Models Using Hadoop Mapreduce, Matthew Shaffer

e-Research: A Journal of Undergraduate Work

In this paper, I will explain how I used the probability modeling tool, Markov Models, in combination with Hadoop MapReduce parallel programming platform in order to quickly and efficiently analyses documents and create a probability model of them. I will explain what Markov Models are, give a brief overview of what MapReduce is, explain why Markov models can be used for document analysis, explain my code of the modeling program, and examine the performance of various MapReduce platforms and techniques in analyzing documents.