Visualizing MapReduce Algorithm with WordCount Example:

Standard

In this blog-post, we would visualize how MapReduce Algorithms operates to perform a Word Count on a Text Input:

First of all, for all programmers out there, Here is the code (Javascript):

var map = function (key, value, context) {
    var words = value.split(/[^a-zA-Z]/);
    for (var i = 0; i < words.length; i++) {
        if (words[i] !== "") {
            context.write(words[i].toLowerCase(), 1);
            }
        }
};
var reduce = function (key, values, context) {
    var sum = 0;
    while (values.hasNext()) {
          sum += parseInt(values.next());
    }
    context.write(key, sum);
};

Courtesy: Microsoft Hadoop on Azure Samples

Now, let’s visualize this using an example.

Suppose the Text is “Hadoop on Azure sample Hadoop is on Windows Azure Hadoop is on Windows server” – Then this is how you can think of what happens to your input when it is processed first by Map function and then by Reduce function:

INPUT MAP REDUCE

Hadoop on Azure sample

Hadoop is on Windows Azure

Hadoop is on Windows server

Hadoop 1 Hadoop 3
On 1
Azure 1 on 3
Sample 1
Hadoop 1 Azure 2
Is 1
On 1 Sample 1
Windows 1
Azure 1 Is 2
Hadoop 1
Is 1 Windows 2
On 1
Windows 1 Server 1
Server 1

Conclusion:

In this blog post, we visualized how MapReduce Algorithm operates for a WordCount Example.

Advertisements

4 thoughts on “Visualizing MapReduce Algorithm with WordCount Example:

Thank this author by sharing the article on social media. If you have any questions or comments, please leave a reply below:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s