Course details

Getting Started with Hadoop: Developing a Basic MapReduce Application

Getting Started with Hadoop: Developing a Basic MapReduce Application


Overview/Description
Expected Duration
Lesson Objectives
Course Number
Expertise Level



Overview/Description

In this Skillsoft Aspire course, discover how to use Hadoop's MapReduce; provision a Hadoop cluster on the cloud; and build an application with MapReduce to calculate word frequencies in a text document. To start, create a Hadoop cluster on the Google Cloud Platform using its Cloud Dataproc service; then work with the YARN Cluster Manager and HDFS (Hadoop Distributed File System) NameNode web applications that come packaged with Hadoop. Use Maven to create a new Java project for the MapReduce application, and develop a mapper for word frequency application. Create a Reducer for the application that will collect Mapper output and calculate word frequencies in input text files, and identify configurations of MapReduce applications in the Driver program and the project's pom.xml file. Next, build the MapReduce word frequency application with Maven to produce a jar file and prepare for execution from the master node of the Hadoop cluster. Finally, run the application and examine outputs generated to get word frequencies in the input text document. The exercise involves developing a basic MapReduce application.



Expected Duration (hours)
1.2

Lesson Objectives

Getting Started with Hadoop: Developing a Basic MapReduce Application

  • Course Overview
  • create and configure a Hadoop cluster on the Google Cloud Platform using its Cloud Dataproc service
  • work with the YARN Cluster Manager and HDFS NameNode web applications that come packaged with Hadoop
  • use Maven to create a new Java project for the MapReduce application
  • develop a Mapper for the word frequency application that includes the logic to parse one line of the input file and produce a collection of keys and values as output
  • create a Reducer for the application that will collect the Mapper output and calculate the word frequencies in the input text file
  • specify the configurations of the MapReduce applications in the Driver program and the project's pom.xml file
  • build the MapReduce word frequency application using Maven to produce a jar file and then prepare for execution from the master node of the Hadoop cluster
  • run the application and examine the outputs generated to get the word frequencies in the input text document
  • idenfity the apps packaged with Hadoop and the purposes they serve and recall the classes/methods used in the Map and Reduce phases of a MapReduce application
  • Course Number:
    it_dshpfddj_02_enus

    Expertise Level
    Beginner