Course details

Hadoop HDFS: Introduction to the Shell

Hadoop HDFS: Introduction to the Shell


Overview/Description
Expected Duration
Lesson Objectives
Course Number
Expertise Level



Overview/Description

In this Skillsoft Aspire course, learners discover how to set up a Hadoop Cluster on the cloud and explore bundled web apps—the YARN Cluster Manager app and the HDFS (Hadoop Distributed File System) NameNode UI. This 9-video course assumes a good understanding of what Hadoop is, and how HDFS enables processing of big data in parallel by distributing large data sets across a cluster; learners should also be familiar with running commands from the Linux shell, with some fluency in basic Linux file system commands. The course opens by exploring two web applications which are packaged with Hadoop, the UI for the YARN cluster manager, and the node name UI for HDFS. Learners then explore two shells which can be used to work with HDFS, the Hadoop FS shell and Hadoop DFS shell. Next, you will explore basic commands which can be used to navigate HDFS; discuss their similarities with Linux file system commands; and discuss distributed computing. In a closing exercise, practice identifying web applications used to explore and also monitor Hadoop.



Expected Duration (hours)
0.9

Lesson Objectives

Hadoop HDFS: Introduction to the Shell

  • Course Overview
  • provision a Hadoop cluster on the cloud using the Google Cloud Platform's Dataproc service
  • identify the various GCP services used by Dataproc when provisioning a cluster
  • list the metrics available on the YARN Cluster Manager app and recognize how it can be useful to monitor job executions
  • recall the details and metrics of HDFS available on the NameNode web app and how it can be used to browse the file system
  • identify the tools of the Hadoop ecosystem which are packaged with Hadoop and recall how they can be accessed
  • configure HDFS using the hdfs-site.xml file and identify the properties which can be set from it
  • compare the hadoop fs and hdfs dfs shells and recognize their similarities to Linux shells
  • explore apps for Hadoop, configure HDFS, work with HDFS shells
  • Course Number:
    it_dshdfsdj_02_enus

    Expertise Level
    Beginner